14:57:59 <RRSAgent> RRSAgent has joined #webrtc
14:58:03 <RRSAgent> logging to https://www.w3.org/2023/01/17-webrtc-irc
14:58:25 <Zakim> Zakim has joined #webrtc
15:58:43 <dom> Meeting: WebRTC January 2023 meeting
15:58:43 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/January_17_2023
15:58:43 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Jan/att-0003/WEBRTCWG-2023-01-17.pdf
15:58:43 <dom> Chairs: HTA, Jan-Ivar, Bernard
16:02:44 <dom> Present+ Henrik, Varun, Cullen, Dom, PatrickRockhill, Youenn, PeterThatcher, TimPanton, MikeEnglish
16:02:57 <dom> Present+ Elad
16:03:02 <dom> Present+ Harald
16:03:17 <dom> Present+ Carine
16:04:01 <dom> Present+ TovePetersson
16:04:05 <dom> Present+ Jan-Ivar
16:04:14 <dom> Present+ BenWagner
16:04:25 <dom> Present+ Florent
16:04:44 <dom> Present+ TonyHerre
16:06:12 <dom> Recording is starting
16:09:29 <dom> Topic: Call for Consensus (CfC) Status
16:09:30 <dom> [slide 8]
16:09:59 <dom> Harald: we've seen support on low latency use cases - seeing consensus
16:10:15 <dom> ... on the Face Detection, there is an objection from Bernard - we'll have to review it and come back
16:10:41 <dom> Youenn: resolving the issues related to low latency use cases would be needed to declare consensus
16:10:47 <dom> Harald: I think we can merge and iterate
16:11:13 <dom> Youenn: it's already in the document - I would prefer we remove the notice "no consensus" once these issues are resolved
16:11:25 <dom> Harald: please mark this as an objection on the list then
16:11:44 <dom> ... More CfC expected
16:12:16 <dom> Topic: -> https://github.com/w3c/webrtc-pc/ WebRTC-pc
16:12:16 <dom> Subtopic: Issue #2795 Missing URL in RTCIceCandidateInit
16:12:16 <dom> [slide 12]
16:13:01 <dom> Youenn: we decided to remove the url from RTCPeerConnectionIceEvent
16:13:15 <dom> ... the dictionary to create that event has a candidate field and an URL field
16:13:21 <dom> ... that second field should probably be removed
16:13:59 <dom> ... Usually, events can be shimmed - not for IceEvent, since you can't create an IceCandidate with an undefined URL
16:14:26 <dom> ... do we want to change this?
16:14:45 <dom> ... two questions then: removing URL from IceEventInit, adding URL to the constructor
16:15:07 <dom> Harald: the URL is useful to identify which candidates come from which servers
16:15:19 <dom> ... a consturctor that can't create all values is problematic for testing
16:15:35 <dom> ... I would like to see that IceCandidate can take a URL to generate those candidates
16:16:06 <dom> Jan-Ivar: no strong opinion; but the fact that the constructor doesn't have a parameter doesn't prevent it being added to the object
16:16:20 <dom> Youenn: right, but this leaves edge cases where this wouldn't work as expected
16:16:24 <dom> RRSAgent, draft minutes
16:16:25 <RRSAgent> I have made the request to generate https://www.w3.org/2023/01/17-webrtc-minutes.html dom
16:17:07 <dom> Henrik: no strong opinion, but also finds strange one of these things can't be constructed
16:17:38 <dom> Present+ Bernard
16:17:55 <dom> RESOLVED: mild preference to add url to icecandidate constructor and consensus to remove url from IceEventInit
16:18:27 <dom> Jan-Ivar: would this mean that an JSON-stringified IceCandiate would include an url?
16:18:43 <dom> Youenn: it's a separate issue
16:19:04 <dom> Jan-Ivar: they're linked given they use the same dictionary
16:19:16 <dom> Youenn: we could define a different dictionary for json-ification
16:19:30 <dom> Jan-Ivar: also needs to consider the impact on addCandidate
16:19:54 <dom> Subtopic: Issue #2780 duplicate rids in sRD underspecified
16:19:54 <dom> [slide 13]
16:20:21 <dom> [merged since no consensus was expressed on github]
16:20:31 <dom> Subtopic: PR #2801: Prune createAnswer()'s encodings and [[SendEncodings]] in sLD(answer)
16:20:32 <dom> [slide 14]
16:20:45 <dom> [merged since no consensus was expressed on github]
16:21:16 <dom> Topic: -> https://github.com/w3c/webrtc-extensions WebRTC Extensions
16:21:16 <dom> Subtopic: Issue #43 / PR #139: Mixed Codec Simulcast
16:21:16 <dom> [slide 19]
16:21:33 <dom> Bernard: we've discussed this at the July meeting
16:21:40 <dom> ... Florent developed PR #139
16:22:06 <dom> ... the use case is for mixed codec simulcast, e.G. you want to use AV1 but will only get decent performant at low resolution
16:22:21 <dom> ... you would use a different codec at a higher res (e.g. vp8 or vp9)
16:23:16 <dom> [slide 20]
16:24:22 <dom> [slide 21]
16:24:55 <dom> Bernard: this example puts 2 codecs in - AV1 and VP8; at full res, only VP8
16:25:55 <dom> [slide 22]
16:25:55 <dom> Florent: issue #126 addresses another problem that this PR would also help cover
16:26:24 <dom> ... some applications want to select which codec is used but without renegotiation reordering
16:26:47 <dom> ... the current approach is heavy, annoying and issue-prone
16:27:01 <dom> [slide 23]
16:28:50 <dom> [slide 24]
16:31:39 <dom> Bernard: questions about weird cases in the field
16:32:06 <dom> ... imagine there is a hardware encoder but is not available because it gets preempted
16:32:26 <dom> ... are there situations where having an array would allow for a better fallback? although this wouldn't help in this case
16:32:56 <dom> Florent: wrt limited capacity of hardware encoder- if there is no osftware fallback, setParameters would throw if resources can't be acquired
16:33:00 <dom> s/osf/sof
16:33:12 <dom> ... although not if that happens later - Henrik has a proposal for that
16:33:42 <dom> ... if you run out of capacity on the hardware encoder, there is no control to surface errors upon software fallback
16:33:57 <dom> Bernard: maybe Henrik's proposal will help there indeed
16:34:36 <dom> Harald: the renegotiation problem is prettily easily solved: when you set the encoding, it must be valid; when you negotiate (even 1st one), you remove anything that isn't in the negotiated codecs
16:34:59 <dom> ... for ease of use, we should have an array and use the 1st entry of the array that are still available after negotiations
16:35:25 <dom> Henrik: I have a proposal that somewhat overlaps with it that we will talk about later
16:35:34 <dom> ... I do have a preference for a single codec value in setParameters
16:35:45 <dom> ... there should either be sensible defaults or have the stream disabled
16:36:00 <dom> ... I would keep this API surface as simple as possible
16:36:15 <dom> Florent: if the selected codec doesn't match, we could throw an error for the app to handle
16:36:56 <dom> jan-ivar: in the API so far, we've tried hard to keep setParameters and negotiation deal with the same settings to avoid creating races
16:37:04 <dom> ... that may be solved by what Harald described
16:37:22 <dom> Florent: the negotation is about what codecs are allowed, not the ones that are used - the usage is not the same
16:37:44 <dom> ... at the moment, renegotiation is used to push the first in the list to get it used, but I think more control is needed
16:37:54 <dom> Bernard: there may be a difference between before and after offer/answer
16:38:23 <dom> ... after offer/answer, the codecs in the list are within the negotiated envelop, you check against that, not capabilities
16:38:49 <dom> ... for addTransceiver, potentially there hasn't been an O/A yet
16:39:22 <dom> ... if you haven't called setCodecPreferences, it could be any in capabilities; this could lead to a contradiction with the addTransceiver
16:39:36 <dom> ... this has to be thought through, probably iterating in the PR
16:40:02 <dom> Florent: we should indeed check against capabilities, codec preferences; we should align with what is done e.g. in SVC
16:40:45 <dom> ... maybe sRD should throw an error; developer tools may help provide more visibility on what SDP would send
16:40:57 <dom> ... maybe we can iterate on this on github as we prepare the PR
16:41:12 <dom> Subtopic: Issue #127: How to deal with encoder errors?
16:41:13 <dom> [slide 25]
16:41:19 <dom> Henrik: somewhat related but also different
16:43:20 <dom> [slide 26]
16:43:26 <dom> RRSAgent, draft minutes
16:43:28 <RRSAgent> I have made the request to generate https://www.w3.org/2023/01/17-webrtc-minutes.html dom
16:44:58 <dom> Bernard: should it always be active=false on all layers? e.g. if only a given encoder is a source of errors
16:45:29 <dom> Henrik: you may want to know which layers the errors happened; the event may need to surface which encoders this error occurred
16:45:41 <dom> jan-ivar: clarification that this is an event, not callbacks
16:46:11 <dom> ... is it necessary to set active to false and let JS deal with the situation overall?
16:46:33 <dom> henrik: if the encoder doesn't work, it can't keep encoding: it needs to be stopped to fallback to a sensible default
16:46:54 <dom> ... I'm concerned that any default would end up sending unexpected keyframes
16:47:30 <dom> youenn: what might constitute an error? e.G. transient vs fatal error? this may lead to fragmentation
16:47:45 <dom> ... do we want to articulate this on error vs not error, or a change more generally?
16:48:18 <dom> henrik: very good point; some errors may simply be a notification but the app may not need to act because it can be recovered from
16:48:44 <dom> ... would be useful to say whether this should include the fallback in case of codec removal from negotiation
16:48:56 <dom> harald: we shouldn't stop anything unless the error forces it
16:49:12 <dom> ... so setting active=false shouldn't only impact affected layers
16:49:18 <dom> s/shouldn't/should/
16:49:47 <dom> ... it should be an event, since events can have a default behavior that can be disabled
16:50:25 <dom> ... so the event could be fired every time there is a significant change, and by default let it managed by the UA that the app can intercept
16:50:43 <dom> Bernard: +1 to Youenn and Harald
16:51:08 <dom> ... but I don't think it's for recovery - this is just for real errors?
16:51:28 <dom> Henrik: right, but there may still be fallbacks (hardware → software)
16:51:43 <dom> bernard: if the error is recoverable, I would assume you wouldn't have that event
16:52:03 <dom> Youenn: some OSes are changing from hardware to software based on the resolution of the stream - not even an error
16:52:14 <dom> henrik: maybe this should be scoped to unrecoverable errors?
16:53:00 <dom> timp: I like this, but I don't think it's about errors, it should be codecavailabilitychange
16:53:03 <dom> henrik: good point
16:53:04 <dom> harald: +1
16:53:19 <dom> Subtopic: Issue #130: how does setOfferedRtpHeaderExtensions work?
16:53:19 <dom> [slide 27]
16:54:36 <dom> Harald: Fippo and I were disagreeing on the interpretation of the spec - I'm now thinking Fippo is right, but want to make sure the WG is also comfortable with that interpretation - please chime in in the issue
16:55:01 <dom> jan-ivar: +1 to fippo's interpretation, which is also more Web compatible
16:55:25 <dom> ... also frozenarrays in dictionary is frowned upon
16:55:34 <dom> youenn: +1 To fippo's as well
16:55:55 <dom> Topic: -> https://github.com/w3c/webrtc-encoded-transform WebRTC Encoded Transform
16:55:55 <dom> [slide 30]
16:56:39 <dom> [slide 31]
16:57:52 <dom> [slide 32]
16:59:06 <dom> [slide 33]
16:59:24 <dom> Harald: I designed an API for frame handling
16:59:31 <dom> ... creating frames from data and metadata
16:59:45 <dom> ... modify a frame metadata (in particular to avoid data copy)
17:00:13 <dom> ... data modification would happen async from metadata - which raises the question about consistency of the frame
17:00:27 <dom> [slide 34]
17:01:05 <dom> Harald: we've been reasonably successful using streams for frames; but reconfiguration requests are more events-like
17:01:09 <dom> [slide 35]
17:01:31 <dom> Harald: I propose an interface to handle this, as previously presented after the IETF hackathon
17:02:30 <dom> [slide 36]
17:03:13 <dom> [slide 37]
17:03:50 <dom> Harald: the long term plan would be to redefine RTPSender / RTPReceiver as composed of smaller components (encoder, packetizer)
17:04:23 <dom> [slide 38]
17:05:35 <dom> Bernard: is there an assumption that the packetizer is the one in the browser, or would it be possible to bring your own packetizer? e.g. would be useful for HEVC in WebRTC
17:06:30 <dom> Harald: there is a limited number of behaviors for packetizers - we should enable these different behaviors, haven't looked at bringing a fully custom packetizers
17:06:41 <dom> Bernard: this may impact the discussion of the use case
17:07:02 <dom> ... another question about workers: in WebCodec, encode/decode would typically happen in a worker
17:07:17 <dom> ... would this imply bringing RTCSender/Receiver in workers?
17:07:35 <dom> Harald: unsure about that one
17:07:59 <dom> ... events aren't transferred
17:08:27 <dom> ... making objections transferable can prove tricky, as we've learned with MediaStreamTrack
17:08:53 <dom> Peter: +1 to considering these use cases in scope and calling for proposals
17:09:15 <dom> ... I would like to get clarity on whether custom packetizer is part of that though, e.g. for custom SVC
17:09:44 <dom> Harald: none of the use cases in my list require custom packetization, so such use cases would need to be added
17:10:29 <dom> ... I'm hesitant and somewhat nervous to expose packet levels to JS, esp without strong supporting use cases
17:10:51 <dom> Bernard: what should be the next steps?
17:11:06 <dom> Harald: run a CfC on use cases? if approved, then we would iterate on proposals
17:12:04 <dom> Jan-Ivar: the use cases could use a bit more specificity; I'm worried about having too many APIs to achieve the same thing
17:12:25 <dom> ... there is already a way to do relay where implementations could optimize decode/encode
17:12:44 <dom> ... although the modification use case is a good illustration of what additional would be needed
17:13:08 <dom> ... I don't see a problem with using streams in events for the control path
17:13:19 <dom> Youenn: +1 to the question wrt packet vs frame
17:13:44 <dom> ... if we want to a packet level API, we'll need to figure out the security model, which will lead to a very different path
17:14:13 <dom> Harald: I haven't yet seen the use case written up that warrants a packet-level API; I'm very happy to see it, discuss it and decide based on it
17:14:28 <dom> ... but at the moment, what I've seen needed is possible to do at the frame level
17:14:38 <dom> ... hence why I'm pursuing it
17:14:39 <dom> ... so let's see the use cases
17:15:23 <dom> Dom: do we want to wait for these additional use cases before CfC these ones?
17:15:44 <dom> Harald: no, I think they can live on their own; not clear that the packet level API would fully address them in any case
17:15:52 <dom> Topic: -> https://www.w3.org/community/sccg/ Screen Capture Community Group
17:15:52 <dom> [slide 41]
17:15:56 <dom> RRSAgent, draft minutes
17:15:57 <RRSAgent> I have made the request to generate https://www.w3.org/2023/01/17-webrtc-minutes.html dom
17:18:12 <dom> Toipc: -> https://github.com/w3c/mediacapture-handle De-adopting Capture Handle Identity
17:18:12 <dom> [slide 44]
17:20:03 <dom> [slide 45]
17:21:15 <dom> Jan-Ivar: the WebRTC WG has been in charge of APIs that produce or consume MediaSTreamTrack
17:21:34 <dom> ... I don't think it would be progress from moving specs from W3C WGs to a CG - it feels like a step backwards
17:21:45 <dom> ... it's been less than a year since we adopted the spec
17:21:59 <dom> ... Capture Handle actions is a supplement to identity, not an alternative
17:22:59 <dom> ... traditionally, we "de-adopt" a spec due to lack of interest; Mozilla is definitely interested in this API, so we don't think it should be de-adopted
17:23:27 <dom> Bernard: procedurally, are you suggesting a CfC to de-adopt Capture Handle?
17:23:46 <dom> Elad: what I want to ahppen is Capture Handle Idendity to be be incubated before being brought back
17:24:10 <dom> ... I think the Screen Capture CG would be the right place - it could be either by delegation or copy
17:24:26 <dom> Bernard: this would be limited to Capture Handle Identity?
17:24:28 <dom> Elad: correct
17:25:34 <dom> Youenn: I don't think the WebRTC WG approval is needed to fork the spec; the CG can do it on its own
17:25:51 <dom> ... I don't see value in removing it from the WebRTC WG
17:27:35 <dom> Jan-Ivar: with regard to disagreements, my view is that they've been minor - there is overall agreement on the direction
17:27:59 <dom> Dom: my preference would be to keep it in the WG since I think the disagreements are not critical
17:28:41 <dom> ... but I think a situation where the specs exist in two places is the worse situation for the community in terms of clarity
17:28:53 <dom> RESOLVED: Start a CfC on de-adopting Capture Handle Identity
17:29:00 <dom> Topic: -> https://github.com/w3c/mediacapture-screen-share/issues/255 Auto-pause Capture
17:29:01 <dom> [slide 47]
17:29:24 <dom> [slide 48]
17:29:56 <dom> [slide 49]
17:31:08 <dom> [slide 50]
17:33:04 <dom> [slide 51]
17:33:51 <dom> [slide 52]
17:34:40 <dom> [slide 53]
17:34:57 <dom> [slide 54]
17:35:40 <dom> Youenn: the use case makes sense and is worth saving
17:35:49 <dom> ... I think the API should be the level at the source, not the track
17:35:57 <dom> ... so probably in the capturecontroller
17:36:34 <dom> ... but we can dive into the API shape once we agree on solving the issue
17:37:04 <dom> TimP: does this cover audio as well? the reasons for pausing seem video orientated
17:37:26 <dom> elad: interesting question; it could support Youenn's point about capturecontroller
17:37:36 <dom> ... or maybe we need a source object as I've been discussing with Ben
17:37:57 <dom> ... it may be worth having separate control for audio & video who are perceived differently
17:38:29 <dom> TimP: this could be used in case your webrtc call gets mic stolen e.g. by a GSM call
17:38:46 <dom> elad: maybe that's already covered by the muted event, would need to look at it
17:38:52 <dom> TimP: let's look at audio in general
17:39:05 <dom> Harald: This use case is definitely worth solving
17:39:23 <dom> ... traditionally we haven't exposed Sources to JS, which is my worry everytime we talk about them
17:39:38 <dom> ... we might want to; maybe the CaptureController is the source
17:40:01 <dom> ... I would like to mention again preventDefault() in the event interface that allows to intercept the event default impact
17:40:40 <dom> Jan-Ivar: +1 to Youenn on bringing this to capturecontroller (which is indistinguishable from a source in a case of a single capture)
17:41:13 <dom> ... I don't think we should terminate output; events should be optional, default shouldn't terminate output
17:41:40 <dom> ... if this doesn't move to capturecontroller, I have other issues (e.g. confusion between muted/unmuted, paused)
17:42:23 <dom> Elad: wrt CaptureController, it makes some sense; but I have worries about transferability given that CaptureController isn't transferable
17:42:28 <dom> ... needs to be evaluated more
17:43:01 <dom> ... With respect to default actions, a core component of the proposal is to preserve the legacy behavior - if an event handler isn't set, the output isn't paused
17:43:37 <dom> jan-ivar: setting an eventhaldner shouldn't have a side effect though
17:44:09 <dom> Elad: slide 53 has a possible approach to this
17:44:30 <dom> Harald: preventDefault will help with that
17:44:45 <dom> Elad: sold
17:45:17 <dom> ... So next steps include evaluating MediaSTreamTrack vs CaptureController vs a new Source object?
17:45:23 <dom> Harald: +1
17:45:31 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/pull/77 MediaStreamTrack Frame Rates
17:45:31 <dom> [slide 57]
17:47:21 <dom> [slide 58]
17:48:49 <dom> TimP: uncomfortable with "decimated" which should relate to a factor of 10, not meant here
17:49:02 <dom> ... what do we think the developer would do with this information?
17:49:19 <dom> Henrik: you can measure deltas between cameras settings and what you're getting
17:49:25 <dom> ... you may reconfigure the camera
17:49:53 <dom> ... also useful for debugging - frames being dropped from camera issues or other issues
17:50:15 <dom> ... right now, hard to make sense of frames dropped
17:50:21 <dom> TimP: so mostly a diagnostic tool
17:50:25 <dom> henrik: yes
17:50:37 <dom> jan-ivar: what happens if track.enabled = false?
17:50:56 <dom> henrik: that needs to be decided - maybe stop incrementing counters
17:51:34 <dom> youenn: I understand delivered, decimated; is the total sum of frames those generated by the cameras?
17:51:46 <dom> ... if so, maybe instead of "dropped", we provide the total as "framesGenerated"?
17:51:52 <dom> henrik: fine with me
17:52:15 <dom> jan-ivar: what happens in low-light conditions?
17:52:31 <dom> henrik: framesGenerated would lower (in this new model)
17:52:59 <dom> henrik: hearing overall support with some proposed changes
17:53:32 <dom> Topic: Wrap up
17:53:38 <dom> Bernard: a number of action items:
17:53:46 <dom> ... - CfC on Harald's use cases
17:53:53 <dom> ... - CfC on de-adoption of capture handle
17:54:38 <dom> Elad: please chime in on the issue for auto pause to help with the next iteration
17:54:45 <dom> Youenn: an event on capturecontroller should suffice
17:55:00 <dom> Elad: maybe more is needed to distinguish audio/video
17:55:17 <dom> ... I'll flesh this out
17:56:37 <dom> RRSAgent, draft minutes
17:56:39 <RRSAgent> I have made the request to generate https://www.w3.org/2023/01/17-webrtc-minutes.html dom
17:56:45 <dom> RRSAgent, make log public
19:23:38 <Zakim> Zakim has left #webrtc