Meeting minutes
Slideset: https://
State of the WebRTC WG 🎬
Harald: We'll review what we're chartered to do
… What we've been doing (not in too many details)
… and what we need to do going forward
Harald: We last rechartered in October 2020, with a focus on forward looking changes
… there is also a bunch of document that needs to get published and maintained - 15 listed in the charter
… the charter pushes us toward starting from use cases for new work
… and look at direct control (inspired by ORTC)
Harald: What we have done within our chartered work:
… We have got WebRTC 1.0 to Rec!!!!
… we're still working on bringing mediacapture-main to Rec
… and have 13 documents that are more or less dormant
<vr000m> Hi, could someone let me in to the Hangout call?
Harald: use cases need more work, incl clarity on what they are and need
… we're also doing maintenance work on webrtc based on the bugs found through experience, with fixes coming in
… in some cases, it's a matter of aligning the spec with existing implementations
Harald: what we've done beyond what the charter envisioned - a number of extensions have been proposed
… they unlock some use cases, but not necessarily the ones we've listed
… some of these proposals have been highly controversial, some a bit, and some not at all
<vr000m> @dom no a different URL. joining the new one, thanks!
Harald: one of my worries is that a lot of the discussions are happening between roughly 3 people, all representing browser vendors
… the mailing list is mostly used for announcements, and the debates happen mostly in github issues
… we've managed to pull feedback from other groups (which is good), but failed to attract input from our own group (bad)
… may be too hard to follow, or not directly relevant to most participants - but that's worrisome in the quality of consensus we're obtaining
Harald: The way we imagined the work would happen: a need emerges, is distilled in a scenario in the use case doc, we build an extension to match, and get consensus on it
Harald: but in practice, the needs tend to emerge "in house"
… the solution gets developed also in house
… which is then brought to the WG
… and then we're struggling in finding consensus, esp fast consensus
… e.g. my 11 months journey on breakout box
… is an illustration on how we don't want the process to go
Harald: Another problem is editing resources
… The goal of W3C process is to reach Recommendation
… that means a spec with consensus, a test suite that exercises that can be tested
… and matching implementations
… We have a lot of documents that are languishing
… they specific interface present (maybe) in some browsers
… but we're missing test suites, implementations, people that address bug fixes
… that is worrisome
… Who need these features? can those that need these features put the time and energy to get the work done?
… If nobody shows up, should we abandon the work?
TimP: in the process you describe we currently use is strongly tied to ability to deploy new browser code
… which reinforces the problem you were describing in terms of involvement
Harald: the only counter-example I can think of is Sergio bringing SVC to Chromium
… it shows it's possible to bring code in Chrome without being a Googler
… but I agree the risk you describe is real
Youenn: 13 documents is a lot, but some are with implementations (getDisplayMedia, mediarecorder)
… they get updated based on implementation feedback
… they're shipped and used, they need to pass the line to go to Rec
… what will be the gain to go to Rec?
… this may explain the lack of motivation
… if we have good enough interop
Harald: one of my example was on media capture depth
Bernard: in IETF, we pared down the process because it was too much work with too little benefit
… the same may be happening here
… due to the different between costs and benefits
… but we need to distinguish the ones that are abandoned and not used, vs those that are not pushed forward
Harald: quite a few are in active use in multiple browsers
… but the presence of years old bugs in these specs make me nervous
Cullen: in the general sense, when the WebRTC effort started, it was seen as a negotiation between implementors and users
… but the W3C Process gives a veto to implementors on every thing (esp given that there isn't so many real different implementations)
… I'm not saying it didn't work in delivering result, but if it has become a one-sided process, it has led to a one-sided conversation
… In terms of the use cases that are needed: we're doing this for developers, but also for end users
… every major rtc apps are moving away from the Web because of the poor UX there
… the biggest reason being likely camera/microphone selection
… we need to look at it holistically
… the Web apps would be better in many ways, but we need to go back to look at the pain points
dom: re reaching Rec, we're in a position to assert how much efforts is worth doing in going to the last stage - but we need people to do to assessment; in terms of process - I don't think the W3C process is one-sided, but if it has worked that way in the group, than obviously we need to improve
TimP: I don't expect vendors to go back to the Web given all the investments in native
… what we should be look at instead is where it isn't used and it should be
… my own view is IoT
… there are just a few things that needs changing that would make it so much easier
… I would love for us solve these issues, mostly around ICE
Jan-Ivar: Companies have been redirecting from Web to apps beyond WebRTC due to other incentives
… and the WebRTC client gives broader reach when the audience doesn't want to install the app of the host
… The fact that we're recording and publishing our meetings may play as a disincentive to participation
Bernard: thanks - we need to move on but we'll need to discuss this more
Media Capture Transform 🎬
[scribe missed part of discussion due to connection issue]
JIB: re audio, one way to move ahead may be to polyfill the API based on audio worklet
… re main thread, both proposals can be made to work on the main thread
youenn: I would say there are 3 proposals, with Jan-Ivar's taking the middle path between the two others
… it's difficult to try to solve everything
… we really need to improve video transform - that's our P1
… if we focus on that, we can go faster
… if we're trying to address the problem of audio or main thread, it will only delay getting to a good solution for the biggest issue of video processing
… I would leave anything but video processing on the side
… the key question is whether to use streams or not - I'm hoping the discussion with the WHATWG Streams editors will give us a good picture of whether Streams is workable
… I wouldn't make a decision before we have that discussion
Bernard: our ongoing work is at the intersection of multiple groups: webcodecs, ml, audio, ...
… we don't always understand how these intersections work
… e.g. how well would media capture transform would work with ML APIs
… that makes it harder to answer the design questions
… when in the end, all the APIs need to work together
Harald: we manage to Stream-based WebRTC Encoded Transform API that was developed at the same time of Web Codecs with a different frame format
… which is now causing issues
… getting interfaces to work together is problematic - cross-participation helps, but it's hard to find people that can do that
Bernard: so what should be our next steps? we've mentioned the joint meeting on the Streams issues
… but about the other aspects?
Jan-Ivar: the poll only indicated major support for Streams; no clear support for audio or main thread
Youenn: before picking stream-based, I think we need to have a clearer picture of streams
… e.g. Dom asked whether it solves more than issue than it created, which to me is still unclear
… a future version of Streams might have these issues solved, which may lead to clear informed answer
… hopefully we can have an answer in the upcoming few days or weeks
Harald: I can wait a few more weeks - but we can decide that we're adopting a stream-based proposal knowing the decision can be revisited if it turns out unworkable
… failure is part of life
… but if we wait too long, there is no path to success
Youenn: we can keep making progress in the github issues
Bernard: my biggest concern with streams is around feedback between stages
… stream backpressure doesn't help with encoder rate control
… Harald argued we needed to feedback stream, but we haven't worked it out yet
Youenn: it might be worth filing an issue - not sure if it needs to be streams based
Jan-Ivar: I feel pretty confident about the upcoming Streams meeting - we're getting positive signals on getting some of the resolutions we want landed in
… In terms of rate control - is it specifically about streams getting in the way?
bernard: probably just not helping
jan-ivar: assuming the stream discussions turns out as I expect, how do we move forward with the two other questions?
Harald: re rate control, my initial design had a stream going the other way for feedback incl rate control
… but I abandoned it because it could only be useful if fully specified
… I thought we had an open bug on it, but can't find it - will open one
… not clear that the backchannel had to be stream-based in retrospect
TimP: if you look at gstreamer, the backchannel isn't done the same way as the forward channel
… it's pub/sub bus
… I think the backchannel needs more flexibility that what streams can give you
Jan-Ivar: one way I would frame the state we're in - we have consensus about exposure in workers, we don't have consensus on main thread
… we could proceed by starting with workers-only - exposing on the main thread would be a simple IDL change
Harald: that's possible; speaking with my Chrome hat, the current API is exposed on the main thread and removing it will take work
Youenn: can we apply the same logic to focusing on video first?
Wrap-up 🎬
Bernard: next steps include meeting with WHATWG
… should we include discussion about the use cases at an upcoming interim?
Harald: more generally, doing triage on our document list
Dom: how can we unblock the shape discussion on mediacapture transform
harald: I'm discussing open issues on Jan-Ivar's proposals that is generally growing on me; but need to check broader support from implementation side
… I've created a label in media capture extensions to help tracking these
Bernard: what about the feedback channel discussion?
Harald: file an issue - I have some thoughts on the topic