Meeting minutes
Recording: https://
Slideset: https://
Call for Consensus (CfC) Status 🎞︎
Harald: we've seen support on low latency use cases - seeing consensus
… on the Face Detection, there is an objection from Bernard - we'll have to review it and come back
Youenn: resolving the issues related to low latency use cases would be needed to declare consensus
Harald: I think we can merge and iterate
Youenn: it's already in the document - I would prefer we remove the notice "no consensus" once these issues are resolved
Harald: please mark this as an objection on the list then
… More CfC expected
WebRTC-pc 🎞︎
Issue #2795 Missing URL in RTCIceCandidateInit 🎞︎
Youenn: we decided to remove the url from RTCPeerConnectionIceEvent
… the dictionary to create that event has a candidate field and an URL field
… that second field should probably be removed
… Usually, events can be shimmed - not for IceEvent, since you can't create an IceCandidate with an undefined URL
… do we want to change this?
… two questions then: removing URL from IceEventInit, adding URL to the constructor
Harald: the URL is useful to identify which candidates come from which servers
… a consturctor that can't create all values is problematic for testing
… I would like to see that IceCandidate can take a URL to generate those candidates
Jan-Ivar: no strong opinion; but the fact that the constructor doesn't have a parameter doesn't prevent it being added to the object
Youenn: right, but this leaves edge cases where this wouldn't work as expected
Henrik: no strong opinion, but also finds strange one of these things can't be constructed
RESOLUTION: mild preference to add url to icecandidate constructor and consensus to remove url from IceEventInit
Jan-Ivar: would this mean that an JSON-stringified IceCandiate would include an url?
Youenn: it's a separate issue
Jan-Ivar: they're linked given they use the same dictionary
Youenn: we could define a different dictionary for json-ification
Jan-Ivar: also needs to consider the impact on addCandidate
Issue #2780 duplicate rids in sRD underspecified 🎞︎
[merged since no consensus was expressed on github]
PR #2801: Prune createAnswer()'s encodings and [[SendEncodings]] in sLD(answer) 🎞︎
[merged since no consensus was expressed on github]
WebRTC Extensions 🎞︎
Issue #43 / PR #139: Mixed Codec Simulcast 🎞︎
Bernard: we've discussed this at the July meeting
… Florent developed PR #139
… the use case is for mixed codec simulcast, e.G. you want to use AV1 but will only get decent performant at low resolution
… you would use a different codec at a higher res (e.g. vp8 or vp9)
Bernard: this example puts 2 codecs in - AV1 and VP8; at full res, only VP8
Florent: issue #126 addresses another problem that this PR would also help cover
… some applications want to select which codec is used but without renegotiation reordering
… the current approach is heavy, annoying and issue-prone
Bernard: questions about weird cases in the field
… imagine there is a hardware encoder but is not available because it gets preempted
… are there situations where having an array would allow for a better fallback? although this wouldn't help in this case
Florent: wrt limited capacity of hardware encoder- if there is no software fallback, setParameters would throw if resources can't be acquired
… although not if that happens later - Henrik has a proposal for that
… if you run out of capacity on the hardware encoder, there is no control to surface errors upon software fallback
Bernard: maybe Henrik's proposal will help there indeed
Harald: the renegotiation problem is prettily easily solved: when you set the encoding, it must be valid; when you negotiate (even 1st one), you remove anything that isn't in the negotiated codecs
… for ease of use, we should have an array and use the 1st entry of the array that are still available after negotiations
Henrik: I have a proposal that somewhat overlaps with it that we will talk about later
… I do have a preference for a single codec value in setParameters
… there should either be sensible defaults or have the stream disabled
… I would keep this API surface as simple as possible
Florent: if the selected codec doesn't match, we could throw an error for the app to handle
jan-ivar: in the API so far, we've tried hard to keep setParameters and negotiation deal with the same settings to avoid creating races
… that may be solved by what Harald described
Florent: the negotation is about what codecs are allowed, not the ones that are used - the usage is not the same
… at the moment, renegotiation is used to push the first in the list to get it used, but I think more control is needed
Bernard: there may be a difference between before and after offer/answer
… after offer/answer, the codecs in the list are within the negotiated envelop, you check against that, not capabilities
… for addTransceiver, potentially there hasn't been an O/A yet
… if you haven't called setCodecPreferences, it could be any in capabilities; this could lead to a contradiction with the addTransceiver
… this has to be thought through, probably iterating in the PR
Florent: we should indeed check against capabilities, codec preferences; we should align with what is done e.g. in SVC
… maybe sRD should throw an error; developer tools may help provide more visibility on what SDP would send
… maybe we can iterate on this on github as we prepare the PR
Issue #127: How to deal with encoder errors? 🎞︎
Henrik: somewhat related but also different
Bernard: should it always be active=false on all layers? e.g. if only a given encoder is a source of errors
Henrik: you may want to know which layers the errors happened; the event may need to surface which encoders this error occurred
jan-ivar: clarification that this is an event, not callbacks
… is it necessary to set active to false and let JS deal with the situation overall?
henrik: if the encoder doesn't work, it can't keep encoding: it needs to be stopped to fallback to a sensible default
… I'm concerned that any default would end up sending unexpected keyframes
youenn: what might constitute an error? e.G. transient vs fatal error? this may lead to fragmentation
… do we want to articulate this on error vs not error, or a change more generally?
henrik: very good point; some errors may simply be a notification but the app may not need to act because it can be recovered from
… would be useful to say whether this should include the fallback in case of codec removal from negotiation
harald: we shouldn't stop anything unless the error forces it
… so setting active=false should only impact affected layers
… it should be an event, since events can have a default behavior that can be disabled
… so the event could be fired every time there is a significant change, and by default let it managed by the UA that the app can intercept
Bernard: +1 to Youenn and Harald
… but I don't think it's for recovery - this is just for real errors?
Henrik: right, but there may still be fallbacks (hardware → software)
bernard: if the error is recoverable, I would assume you wouldn't have that event
Youenn: some OSes are changing from hardware to software based on the resolution of the stream - not even an error
henrik: maybe this should be scoped to unrecoverable errors?
timp: I like this, but I don't think it's about errors, it should be codecavailabilitychange
henrik: good point
harald: +1
Issue #130: how does setOfferedRtpHeaderExtensions work? 🎞︎
Harald: Fippo and I were disagreeing on the interpretation of the spec - I'm now thinking Fippo is right, but want to make sure the WG is also comfortable with that interpretation - please chime in in the issue
jan-ivar: +1 to fippo's interpretation, which is also more Web compatible
… also frozenarrays in dictionary is frowned upon
youenn: +1 To fippo's as well
WebRTC Encoded Transform 🎞︎
Harald: I designed an API for frame handling
… creating frames from data and metadata
… modify a frame metadata (in particular to avoid data copy)
… data modification would happen async from metadata - which raises the question about consistency of the frame
Harald: we've been reasonably successful using streams for frames; but reconfiguration requests are more events-like
Harald: I propose an interface to handle this, as previously presented after the IETF hackathon
Harald: the long term plan would be to redefine RTPSender / RTPReceiver as composed of smaller components (encoder, packetizer)
Bernard: is there an assumption that the packetizer is the one in the browser, or would it be possible to bring your own packetizer? e.g. would be useful for HEVC in WebRTC
Harald: there is a limited number of behaviors for packetizers - we should enable these different behaviors, haven't looked at bringing a fully custom packetizers
Bernard: this may impact the discussion of the use case
… another question about workers: in WebCodec, encode/decode would typically happen in a worker
… would this imply bringing RTCSender/Receiver in workers?
Harald: unsure about that one
… events aren't transferred
… making objections transferable can prove tricky, as we've learned with MediaStreamTrack
Peter: +1 to considering these use cases in scope and calling for proposals
… I would like to get clarity on whether custom packetizer is part of that though, e.g. for custom SVC
Harald: none of the use cases in my list require custom packetization, so such use cases would need to be added
… I'm hesitant and somewhat nervous to expose packet levels to JS, esp without strong supporting use cases
Bernard: what should be the next steps?
Harald: run a CfC on use cases? if approved, then we would iterate on proposals
Jan-Ivar: the use cases could use a bit more specificity; I'm worried about having too many APIs to achieve the same thing
… there is already a way to do relay where implementations could optimize decode/encode
… although the modification use case is a good illustration of what additional would be needed
… I don't see a problem with using streams in events for the control path
Youenn: +1 to the question wrt packet vs frame
… if we want to a packet level API, we'll need to figure out the security model, which will lead to a very different path
Harald: I haven't yet seen the use case written up that warrants a packet-level API; I'm very happy to see it, discuss it and decide based on it
… but at the moment, what I've seen needed is possible to do at the frame level
… hence why I'm pursuing it
… so let's see the use cases
Dom: do we want to wait for these additional use cases before CfC these ones?
Harald: no, I think they can live on their own; not clear that the packet level API would fully address them in any case
Screen Capture Community Group 🎞︎
Toipc: De-adopting Capture Handle Identity
Jan-Ivar: the WebRTC WG has been in charge of APIs that produce or consume MediaSTreamTrack
… I don't think it would be progress from moving specs from W3C WGs to a CG - it feels like a step backwards
… it's been less than a year since we adopted the spec
… Capture Handle actions is a supplement to identity, not an alternative
… traditionally, we "de-adopt" a spec due to lack of interest; Mozilla is definitely interested in this API, so we don't think it should be de-adopted
Bernard: procedurally, are you suggesting a CfC to de-adopt Capture Handle?
Elad: what I want to ahppen is Capture Handle Idendity to be be incubated before being brought back
… I think the Screen Capture CG would be the right place - it could be either by delegation or copy
Bernard: this would be limited to Capture Handle Identity?
Elad: correct
Youenn: I don't think the WebRTC WG approval is needed to fork the spec; the CG can do it on its own
… I don't see value in removing it from the WebRTC WG
Jan-Ivar: with regard to disagreements, my view is that they've been minor - there is overall agreement on the direction
Dom: my preference would be to keep it in the WG since I think the disagreements are not critical
… but I think a situation where the specs exist in two places is the worse situation for the community in terms of clarity
RESOLUTION: Start a CfC on de-adopting Capture Handle Identity
Auto-pause Capture 🎞︎
Youenn: the use case makes sense and is worth saving
… I think the API should be the level at the source, not the track
… so probably in the capturecontroller
… but we can dive into the API shape once we agree on solving the issue
TimP: does this cover audio as well? the reasons for pausing seem video orientated
elad: interesting question; it could support Youenn's point about capturecontroller
… or maybe we need a source object as I've been discussing with Ben
… it may be worth having separate control for audio & video who are perceived differently
TimP: this could be used in case your webrtc call gets mic stolen e.g. by a GSM call
elad: maybe that's already covered by the muted event, would need to look at it
TimP: let's look at audio in general
Harald: This use case is definitely worth solving
… traditionally we haven't exposed Sources to JS, which is my worry everytime we talk about them
… we might want to; maybe the CaptureController is the source
… I would like to mention again preventDefault() in the event interface that allows to intercept the event default impact
Jan-Ivar: +1 to Youenn on bringing this to capturecontroller (which is indistinguishable from a source in a case of a single capture)
… I don't think we should terminate output; events should be optional, default shouldn't terminate output
… if this doesn't move to capturecontroller, I have other issues (e.g. confusion between muted/unmuted, paused)
Elad: wrt CaptureController, it makes some sense; but I have worries about transferability given that CaptureController isn't transferable
… needs to be evaluated more
… With respect to default actions, a core component of the proposal is to preserve the legacy behavior - if an event handler isn't set, the output isn't paused
jan-ivar: setting an eventhaldner shouldn't have a side effect though
Elad: slide 53 has a possible approach to this
Harald: preventDefault will help with that
Elad: sold
… So next steps include evaluating MediaSTreamTrack vs CaptureController vs a new Source object?
Harald: +1
MediaStreamTrack Frame Rates 🎞︎
TimP: uncomfortable with "decimated" which should relate to a factor of 10, not meant here
… what do we think the developer would do with this information?
Henrik: you can measure deltas between cameras settings and what you're getting
… you may reconfigure the camera
… also useful for debugging - frames being dropped from camera issues or other issues
… right now, hard to make sense of frames dropped
TimP: so mostly a diagnostic tool
henrik: yes
jan-ivar: what happens if track.enabled = false?
henrik: that needs to be decided - maybe stop incrementing counters
youenn: I understand delivered, decimated; is the total sum of frames those generated by the cameras?
… if so, maybe instead of "dropped", we provide the total as "framesGenerated"?
henrik: fine with me
jan-ivar: what happens in low-light conditions?
henrik: framesGenerated would lower (in this new model)
henrik: hearing overall support with some proposed changes
Wrap up 🎞︎
Bernard: a number of action items:
… - CfC on Harald's use cases
… - CfC on de-adoption of capture handle
Elad: please chime in on the issue for auto pause to help with the next iteration
Youenn: an event on capturecontroller should suffice
Elad: maybe more is needed to distinguish audio/video
… I'll flesh this out