Meeting minutes
Rechartering update
tidoust: AC review ongoing. Please review the draft charter and vote on it. The more support the better.
TPAC 2021 planning
cpn: TPAC is W3C's annual "F2F" meetings. Proposal is for TPAC to be organized over 2 weeks: one week for joint meetings, one week for breakout sessions.
… One planned joint meeting with WebRTC WG, e.g. to discuss WebCodecs.
… I suggest to focus TPAC on joint meetings.
Tess: If the group can do 1-2 day during that week, that's fine with me. I would recommend against scheduling a group meeting one week before that otherwise TPAC spans too many weeks in a row.
chcunningham: Happy to focus on joint meeting given the virtual nature of the meeting.
cpn: I can say that the Media & Entertainment IG will want to organize a joint meeting. That's me with another chair hat on.
… Open for suggestions otherwise!
jernoble: I just want to make sure that everyone gets a chance to be heard. If you want to meet, please say so!
Welcome Alastor Wu as editor for AutoPlay Policy Detection
jernoble: Alastor is not around today, but thank you for volunteering for this job! Looking forward to discussing Autoplay Policy Detection.
cpn: Alastor raised some questions on the spec. We'll need feedback from other implementers.
jernoble: Looking into it.
chcunningham: We'll discuss it internally and get back to you.
dale: You can use me as a contact in the meantime.
Codec switching in MSE & FPWD
tidoust: The pending pull request means we've added the main feature the group is set to work on for MSE. Can we issue a CfC to publish FPWD once the PR lands?
jernoble: what's the plan for MSE on the Rec track?
tidoust: If there are additional features you think should be in the FPWD, we can wait. FPWD starts a call for exclusions, so it's a good idea to have the features you're planning to have in there. It does not have to be perfect.
jernoble: MSE in workers has already landed in Chrome. Also playback through gaps in unbuffered ranges. That work has already been done. If it's ready to go, we can start the publication process
tidoust: In that case I'd suggest postponing until those features are specified
dale: Matt's timeline is around June 30th, should have more to say next meeting. We're working towards an origin trial for MSE in workers
jernoble: A good opportunity to tie it all together for the FPWD
WebCodecs issues
jernoble: 3 outstanding issues on WebCodecs
… Design questions. Want to make sure that everyone gets heard and see whether we can come to consensus.
Should WebCodecs be exposed in Window environments?
Should WebCodecs be exposed in Window environments? (#211)
chcunningham: WebCodecs are currently exposed on Window and workers
… WebCodecs does not do heavy lifting on the main thread.
… It was intuitive to us that this got exposed to window.
… The points raised by Youenn are about performance concerns: if main thread is with lots of contention, it becomes problematic to queue and manage your inputs and outputs.
… We think that's why workers are a good fit for many applications.
… That's not the case for all applications though.
… There are applications that are not latency-sensitive or that have to deal with main thread contention.
… During last call, it was suggested that we polled out origin trial users.
… Many of them didn't have an opinion. Those who expressed opinions favored exposure to window.
… They reported on applications without performance constraints.
… Additional points: workers add complexity. Also integration with APIs that are not currently exposed to workers. This may be the case in the future but it is a pain for developers in the meantime to integrate with these APIs.
… We did hear from the TAG yesterday that, following the dev survey, they found that it was good to expose to window after all.
Youenn: My perspective is that it is more on what we should prioritize.
… FWIW, we had a very similar discussion in the WebRTC WG about MediaStreamTrack.
… It's not finished. My understanding is that we are leaning towards "worker support is great, let's do that first. Window support is second target. We want to focus on worker and optimize for that."
… With WebCodecs, I'm mostly focusing on video codecs. It's very similar to MediaStreamTrack.
… We know that we won't regret exposing to workers.
… We know that, initially, some people will have to create a JS shim with postMessage and the like, which is a bit sad, but it will push other APIs to workers down the line.
… Moving more stuff out of the main thread is a good thing.
… Some people are saying that there is no benefit so don't want to pay for the added complexity.
… That's not entirely true. There will be repetitive tasks in the main thread. There will be GC calls. If you push to workers, you can better handler user events in the main thread.
… UAs can also decide to lower the priority of transcoding workers that run in the background. More difficult to do if transcoding is running in the main thread.
… In other words, end users may benefit from the additional work.
… That's why I believe that we should push them towards that. We may not want to mandate that in the end.
… Avoid situations where devs say that "it works on my machine" and believe that it's going to work everywhere.
… Start with things we won't regret.
… [WebRTC example]
… This is the world I want people to live in, and the reason I'm pushing for exposing WebCodecs to workers
… Media capture track processing in web??.io. We got that feedback from an xCloud dev: "all your examples should be in workers". Main thread demos are going to be copied and pasted, leading to bad design. If you prototype things in window, you may remain in window, that's my fear.
chcunningham: Youenn wants to wait to see if this flies. My perspective is that the trial already demonstrates that, as evidenced by feedback received. They found no significant thread contention.
… They tried workers as well and found no real improvement.
… Example of game engine that does decode and render. Performance was good as well, and worried they might have to switch to workers.
… Did not quite follow the point on GC.
… Codec work could be de-prioritized too if main thread is contended.
… Agree that the spec or doc should highlight API use in workers
Youenn: Re. de-prioritizing, you will need to keep the queue, so that's difficult. It could be done with changes.
… I think we can quickly find consensus and move forward with workers.
… Less sure about exposition to window.
… Given where we're at with browsers stacks today, some things that could improve the situation are not there yet.
… Re. GC, easier to optimize in a worker because you fully control the script. In the main thread, you might not own the entire JS, so more difficult to control when the GC is going to kick in.
… [Mentions MediaRecorder as a way to do encoding in the main thread for people who do not want to use workers]
… We already have some APIs for decoding audio on the main thread. Not perfect for sure. Instead of adding a new API, we could improve them.
… WebCodecs is going to be successful in low-latency realtime use cases. It may shine in other use cases, but we should get it right in workers first.
Eugene: About low-latency use cases, as I see it, having WebCodecs only on workers increases latency. It creates extra-hops between window and workers. Our colleagues from Stadia were concerned that input APIs are not available in workers, so they'd need to inject extra-jumps.
Youenn: Google Stadia uses WebRTC right now, I believe. Not at a frame-by-frame level. Why the move to WebCodecs?
… You have data channel in the worker. You pipe it to WebCodecs, then you get a frame, then piping into rendering.
… If you need to go through network or something else, we're missing something.
Eugene: WebCodecs being a lower level, Google Stadia is exploring and prototyping.
… I don't want to go into details for them, but that's kind of patronizing for us to assert right ways to do things for them and others.
… Different designs could be working for them.
… It's not only about decoding and outputing stuff.
jernoble: It seems like progress could be made by having specific examples about pages that use WebCodecs, to promote move to workers.
… There is specification experience in doing things in workers not to contend the main thread, typically for audio rendering.
… We used to have bits in main thread, and had to backtrack as it was found to be bad design.
chcunningham: WebCodecs exposes decoded audio, but main use case will create a jitter buffer, etc.
jernoble: Re. "patronizing", we're trying to get the best end-user experiences. I don't believe that it is us patronizing.
… We're not intending to patronize people and tell them how to do their job. We're trying to create best UX for end users.
… We may be able to help with complexity by writing demonstration cases that show how to do things in workers.
… If we are intending to use this API on the window, one thing that this API could improve is providing queueing hooks.
… so that you can propose multiple frames to the decoder so that it always has something to chew on.
chcunningham: I don't follow the "queueing makes GC harder" argument. The queue is an internal detail at that point.
… Re. GC on window not owning the full script, I feel we're talking past each other. The use cases I have in mind own the JS, they know exactly all the bytes that are going on there.
… No objection to extending MediaRecorder. But extending it to do things that WebCodecs would do if it were exposed to window seems a bit strange.
… WebCodecs does not have the same requirements in terms of timing for audio.
… Re. priority of constituencies, I get that but sites also consider themselves as agents for end users and most are doing a good job at making sure performance and UX are good.
… TAG was sympathetic to Youenn's points initially, but now we've heard from the users, and TAG is favouring window exposure.
… That speaks to my previous points: we've heard that worker is not enough.
jernoble: Sites don't have the ability to control whatever happens in the main thread. Other pages may be running on the same process.
… The user is the person using the computer. It is our job to be opiniated about this.
… I don't think using the terms you and Eugene have been using is useful.
cpn: What do we do next? It seems some higher-bandwith conversation before next monthly call would be useful.
hober: There are strong opinions. I'm sad I wasn't in the TAG breakout where this got discussed.
… Re. "objections have been considered", in a WG like this that runs on consensus, we need to establish consensus first before we move on.
chcunningham: I agree that we do not have consensus for now.
… I would like to suggest that we issue a call for consensus on this issue. The points raised today were mostly a summary of the points raised in GitHub. I feel that we're going in circle.
… We have heard from developers. We have heard from the TAG.
… Would it be possible to issue a call for consensus on this?
cpn: My understanding is that a CfC will bring forth statements of positions. We'll establish what we already know that there is not yet a consensus.
… The problem is how do we resolve that?
… The W3C process talks about following the path that creates the least dissension. Perhaps that could be the path we follow.
… If we expose the API to both window and workers, devs will be guided by examples. If they are good, that could be enough.
… It seems to me that there is another WebRTC angle to this that we could perhaps discuss as this seems to originate from there.
Youenn: Yes, it is true that it follows from another WebRTC issue.
chcunningham: Re. CfC, I don't expect it to lead to consensus. The process would do two things: it would make positions explicit, and it forces the chairs to break a tie.
jernoble: I don't think that it is the role of the Chairs to break a tie. We're here to encourage consensus.
jernoble: This conversation is a bit narrow, others should weigh in.
… It may be useful to find other solutions to concerns raised here, such as good practices, examples, other API features that could solve them without removing the API from window.
cpn: I would forward putting forward a proposed resolution along these lines: with supporting documentation, we would move forward with window and worker. We'd be guiding towards most appropriate usage.
… Also supportive of running another meeting. Two other issues to discuss.
chcunningham: I personally would be fine with that proposal. Happy to have another meeting.
jernoble: OK, cpn and I, let's figure that out ASAP.
… Let's document additional points that we discussed today in the GitHub API.
… Apologies for running overtime!
… Hope to see everyone in the next conference