14:54:45 RRSAgent has joined #webrtc 14:54:49 logging to https://www.w3.org/2023/05/16-webrtc-irc 14:54:49 Zakim has joined #webrtc 14:54:49 Meeting: WebRTC May 2023 meeting 14:54:49 Agenda: https://www.w3.org/2011/04/webrtc/wiki/May_16_2023 14:54:49 Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Apr/att-0001/WEBRTCWG-2023-04-18.pdf 14:54:49 Chairs: HTA, Jan-Ivar, Bernard 14:54:56 RRSAgent, make log public 14:58:02 jtsiskin has joined #webrtc 15:00:49 Present+ JaredSiskin, TonnyHerre, FlorentCastelli, TimPanton, HaraldAlvestrand, PeterThatcher, Dom 15:01:27 Present+ PhilippHancke 15:01:59 Present+ BernardAboba 15:02:14 Present+ HenrikBostrom 15:02:57 Recording is starting 15:02:57 Recording: https://www.youtube.com/watch?v=XqYcdxWvlVw 15:03:20 Present+ JanIvarBruaroey 15:03:27 Present+ YouennFablet 15:03:54 Present+ SunSHin 15:04:00 scribe+ 15:05:17 Present+ EladAlon 15:05:28 Topic: -> https://github.com/w3c/webrtc-nv-use-cases WebRTC-NV Use Cases 15:05:28 [slide 10] 15:05:47 TimPanton: we're renaming the "NV" Use Cases into "WebRTC Extended Use Cases" 15:06:02 ... questions were raised about its usefulness, which I think are worth discussing 15:06:16 ... looking for guidance and some sort of agreement of possible improvements 15:06:18 [slide 11] 15:07:02 -> https://www.rfc-editor.org/rfc/rfc7478 RFC7478: Web Real-Time Communication Use Cases and Requirements 15:07:08 Present+ CarineBournez 15:08:04 [slide 12] 15:08:37 TimP: "NV" is no longer what we're doing, so we renamed it into "WebRTC Extended Use Cases" 15:08:48 ... different level of consensus across use cases and requirements 15:09:16 [slide 13] 15:09:55 [slide 14] 15:11:05 Bernard: the Machine Learning use case is a weird one - it does raise a question of what use cases are for 15:11:13 ... in IETF, we distinguish applicability docs from use cases doc 15:11:32 ... the applicability tells you things the standard doesn't apply for 15:11:56 ... documenting that something can be done with the tech would not show up in use cases, but in applicability doc 15:12:10 TimP: worth discussing - will be covered somewhat later 15:12:12 [slide 15] 15:12:26 [slide 16] 15:13:04 [slide 17] 15:13:43 TimP: WISH is addressing 3.9 and 3.10, and yet the use cases don't have consensus 15:13:47 [slide 18] 15:14:29 [slide 19] 15:16:11 [slide 20] 15:17:47 [slide 21] 15:18:38 [slide 22] 15:18:58 TimP: proposal to rename it is in process 15:19:16 ... I want to focus on things that we can only or best do with WebRTC - that implies refocusing on P2P 15:19:27 ... since that's what WebRTC uniquely does 15:19:32 ... similar to RFC7478 did 15:19:47 ... we should take out use cases & requirements done by other standards 15:20:06 ... I think we need to include use cases that don't have new requirements but extend what 7478 describe 15:20:11 ... e.g. IoT 15:20:42 ... We should remove use cases or requirements that don't get consensus in a few months - they can always be added back 15:21:04 ... we should remove use cases that otherwise don't add new requirements 15:21:41 ... proposed API changes should all include changes to the use case doc if there is no use case for it - otherwise, why are we changing the API? 15:21:48 ... this also raises the question of the relationship between explainers and the use case doc 15:22:16 ... we've also been struggling with where the input comes from; happy to use webrtc.nu to that end if that's of interest 15:22:49 Bernard: good suggestions - my opinion on them 15:23:11 ... +1 to the important of P2P; this has been a point of confusion in other WGs use cases 15:23:21 ... Streaming use cases often require P2P operations 15:23:40 ... All of the major cloud streaming services use WebRTC also for P2P among clients 15:24:10 ... "met by other standards" - I would like to see wide usage; WebRTC often wins over other standards because of its reach 15:24:28 ... re removing non-consensus content, +1 15:25:06 ... A big question: are we implying that we need to give blessing to a use case to make it "valid"? 15:25:44 TimP: I think for a small developer, getting some level of reassurance that your usage is available by accident or something that can be relied on makes a big difference 15:26:01 ... as illustrated in the quote in slide 21 15:26:26 Bernard: but even if you put in the doc - it doesn't create a requirement for browsers to support it properly 15:27:01 TimP: True, but when somebody comes up with a change to the API that removes the ability to do this, there is no structural way to push back on the change 15:27:35 Harald: one of the thing that surprises me in this handling of the doc is the handling of consensus 15:27:54 ... e.g. the "one way use case" - the developers are eager to use it, but the WG is pushing back on consensus 15:28:13 ... likewise for trusted JS conferencing - everyone is doing it, but we removed it for lack of consensus 15:28:31 ... I'm worried by the distance between the use case doc and the use cases that I need to support in the real world 15:29:20 Peter: +1 to getting our use cases in order, +1 to having a place where devleopers can ask for things and get a status on "yes, it is in", "yes, it will be soon", "we can't figure out how" 15:29:33 TimP: asking for things shouldn't be asking for features 15:29:48 Jan-Ivar: +1 this needs a clean up 15:29:57 ... we should be clear about the scope for this - this is only for this WG 15:30:10 ... NV was supposed to be WebRTC 2.0 15:30:36 ... instead, we've been unbundling WebRTC in other specs - which make some of these use cases are no longer in scope for our WG 15:30:51 ... the purpose of this doc is to drive discussions for our WG and in our github 15:30:58 ... to help decide what's in or out 15:31:58 Tony: hope we don't silo P2P vs alternatives; there may be use cases for integration with other standards 15:32:14 Youenn: in terms of scope, +1 to Jan-Ivar - this doc is for the WebRTC WG 15:32:33 ... +1 to harald that we haven't had a lot of success with it 15:33:00 ... explainers allow to combine use cases requirements, API and examples - that provide a better structure from my perspective 15:33:05 Present+ GuidoUrdaneta 15:33:18 Youenn: maybe we should migrate towards that more than a standalone use cases doc 15:33:28 TimP: What remains in the use case doc then? 15:33:40 ... is it a list of pointers to use cases in explainers? 15:34:04 ... I'll take this to the list - I can put some time on this once I understand what we want to achieve 15:34:17 ... I want to expand the scope a bit more than just for the WG 15:34:34 Bernard: I'll try to create PRs (esp removal PRs) for our call next month 15:34:56 Topic: -> https://github.com/w3c/webrtc-extensions WebRTC Extensions 15:34:56 Subtopic: Issue #134 / PR #164: Remove JSEP Modifications 15:34:56 [slide 25] 15:36:48 Harald: the RFC errata process is not the process to make change to an IETF document; only the IETF process does that 15:37:05 ... now that the IETF process has turned the crank with the updated draft, we're in a better situation 15:37:09 [slide 26] 15:37:17 Subtopic: Issue #158 / PR #167: Requesting a key frame via setParameters 15:37:18 [slide 26] 15:38:36 [slide 27] 15:40:42 Jan-Ivar: I see a requestFrame boolean - a bit odd for a parameter; why not making it a method? maybe it could be a counter? 15:41:06 Fippo: I didn't want to separate in 2 methods as it creates all kind of issues 15:41:17 ... e.g. when deactivating a simulcast layer 15:41:49 Jan-ivar: this could be done with a Promise.all (vs a boolean attribute); but that's bikeshedding 15:42:01 Fippo: it could cause 2 keyframes to be generated 15:42:12 TimP: are there situations where setParameters isn't the right place for it? 15:42:24 ... e.g. in encryption situations 15:42:34 Fippo: that's why it has been added to encoded transform 15:42:45 ... but we haven't needed it so far for E2EE 15:43:06 Harald: I don't like setParameters setting something that isn't a parameter 15:43:23 ... I would prefer something like sLD 15:43:32 ... getParameters should return what setParameters set 15:43:41 ... requestKeyFrame shouldn't be set 15:43:52 Fippo: maxBitRate is typically unset in the first call of setParameter 15:44:01 Harald: yes, but once it's set, it's set 15:44:41 Florent: I understand the need to have it synchronized with setParameters, but I'm not sure we want it in the encoding parameters 15:44:51 ... we might want to have another member in the structure passed to setParameters 15:45:01 ... that may also make it work for getParameters 15:45:12 ... that would require rid validation - not sure that would necessarily be hard to do 15:45:26 ... the symetric API on a receiver would be very different since we don't have setParameters on receivers 15:45:35 ... how would we handle this there? 15:45:48 ... We do have getParameters on receivers 15:46:09 Fippo: there are use cases for this indeed, e.g. to adjust jitter buffer 15:46:47 Henrik: I support this use case 15:47:09 Jan-Ivar: just because the browser isn't doing the right thing doesn't necessarily imply that it needs to be exposed to JS 15:47:17 ... e.g. it could be done automatically when active is set 15:47:38 Youenn: in encoded transform, there is sender/receiver API, and a transformer API 15:47:43 ... they solve different issues 15:47:49 ... this one is mostly targeting the sender API 15:47:55 ... it makes sense to have it sync'd with setParameters 15:48:20 ... the transform API would still remain, right? 15:48:29 Fippo: it's probably necessary for some E2EE use cases 15:48:59 Jared: this makes sense to me - two reasons to deactivate the top layer: the person left the call or or they're switching down 15:49:19 ... you'll almost always want to send a key frame when deactivating a higher layer, except for the last participant 15:49:35 ... this may not require an API change for most cases 15:49:44 Present+ PatrickRockhill 15:49:55 Peter: I think this is a good approach to solve the real-world need in a simple way 15:50:46 Resolved: refine PR #167 based on input during the call 15:50:59 JIB: I'd like to explore whether we need an API at all 15:51:04 Subtopic: Issue #159: RTCRtpEncodingParameters: scaleResolutionDownTo 15:51:04 [slide 28] 15:51:49 [slide 29] 15:53:19 Peter: what happens if you set the top layer to 360, and the next layer scale down by 2? 15:53:31 Henrik: if you mix the two, we should throw an exception 15:53:41 Peter: so either a factor, or a specific value 15:53:48 ... do we need any specific value? 15:54:05 Henrik: same as for scaleResolutionBy - forbidding upscaling 15:54:35 Peter: what happens when the top layer gets dropped? 15:54:48 Henrik: generally, do the same as SRDB 15:55:13 Youenn: are there existing workarounds with setting active to false? 15:55:25 ... may create short glitch 15:55:37 Henrik: there are workarounds, but I'm not sure how good they are in practice 15:55:52 Florent: an API like this would be great 15:56:10 ... there is a problem with orientation (portrait vs landscape) in terms of interpreting that value 15:56:37 ... also, as Peter asked, what happens if you feed frames that are smaller into layers that expect bigger frames 15:56:56 ... this may result in several layers with a single lower resolution 15:57:20 Henrik: should I provide a PR? 15:57:35 Youenn: I would want to be careful if this adds a lot of complexity 15:57:36 RESOLVED: Overall support in the direction, but details need to be fleshed out and complexity assessed 15:57:42 Topic: -> https://github.com/w3c/mediacapture-extensions Media Capture Extensions 15:57:42 Subtopic: Issue #98 track.getFrameStats() allocates memory, adding to the GC pile 15:57:42 [slide 30] 15:58:22 [slide 31] 15:59:09 [slide 32] 16:00:09 [slide 33] 16:00:44 [slide 34] 16:01:04 [slide 35] 16:01:27 [slide 36] 16:01:59 [slide 37] 16:02:31 [slide 38] 16:03:11 Youenn: the use case is to get stats, not real-time information 16:03:26 ... that's why you call an API, you get a result; if you need it 10s later, you call it again 16:03:46 ... with an interface, you get the object and then the UA starts the processing whether or not you're going to get data from it 16:03:53 ... that's more appropriate for real-time usage 16:04:08 ... it seems an ill-fit for the usage implied by the name getStats 16:04:15 ... so I would go with getStats() 16:04:26 ... if we need real-time info later, they can come through different APIs 16:04:46 ... WebRTC has already this pattern: getStats vs requetsVideoframeCallback 16:04:50 ... so +1 to Proposal A 16:05:13 Jan-Ivar: unfortunately Paul couldn't join us today 16:05:31 ... Paul commented on the issue about IPC - the data has to be in the content process eventually 16:05:44 ... "real-time" is a spectrum (e.g. audio vs graphics) 16:06:06 ... this WG has fallen into a pattern - it would be useful to compare e.g. with the Media WG; not everything has to be a stat 16:06:26 ... there are ways to implement this without locking (which the design guide asks not do in a getter) 16:06:40 ... it all depends on the use case 16:07:01 ... other WGs with real-time have APIs already - media.currentTime or audioCtx.outputLatency 16:07:33 Henrik: the use case is to call it at 1Hz 16:07:44 ... I fail to see use cases to call it more frequently on the main thread 16:08:00 ... if we're confident we're not getting other metrics, this would be fine 16:08:13 ... I don't see a lot of ergonomics difference in using await for async 16:08:27 ... if we do get requests for additional metrics, we would have painted us in a corner 16:08:47 Youenn: with a promise-based API, it's clear that the results will be gathered asynchronously 16:09:17 ... with a getter, it requires updating it very frequently or be specific on the update frequency in the spec 16:09:35 ... the contract is clearer with async 16:09:41 Jan-Ivar: "we should make decision out of love not out of fear" 16:10:08 ... i.e. based on the best information we have now, not anticipating all future uses 16:10:38 Youenn: if we have to design it as a real-time API - would the sync API still be the best choice? 16:10:50 ... we don't have a real strong use case to guide us 16:11:04 Jan-Ivar: a variant of Proposal B is to have the attribute directly on the track 16:11:25 s/tribute/tributes/ 16:11:46 Henrik: it falls down to async vs sync; could you describe what you meant by "lockless"? 16:12:08 JanIvar: there is code that allows reading values from another process without a lock 16:12:34 ... Paul would be the expert on this - he has implemented this for AudioContext (not yet landed in Firefox) 16:13:21 Harald: implementing it as proposal A seems faster and easier 16:13:50 Henrik: not detecting consensus at this point 16:14:37 Jan-Ivar: I'm not hearing a lot of interest in our attribute-based API - will discuss internally and get back to the group on this 16:14:41 Topic: IceController: Prevent removal of candidate pairs w3c/webrtc-extensions#166 16:14:41 [slide 42] 16:15:35 Present+ SameerVijaykar 16:16:01 s/42/41 16:16:04 [slide 42] 16:16:33 [slide 43] 16:17:31 [slide 44] 16:18:54 [slide 45] 16:19:58 [slide 46] 16:21:17 [slide 47] 16:23:27 Peter: this makes sense as a first item to tackle - although it's more useful once combined with candidate pair selection 16:23:43 ... I like the "removable" attribute and a setRemoval method 16:24:09 ... the idleTimeout is interesting, but there may be underlying complexity there; also the ergonomics of max integer value isn't great 16:24:34 Harald: removing candidates isn't timer driven - candidates are removed when they fail 16:24:56 ... the end-of-candidate spec'd in ICE would throw away everything when we reach completed 16:25:05 ... ICE implementations wouldn't fit well with a timeout 16:25:21 ... because of this, cancelable event is a better fit - it allows the ICE Agent to match the spec'd protocol 16:25:53 Jan-Ivar: not an ICE expert - but I prefer preventDefault() 16:26:13 ... still not clear to me what the UA should done once it's prevented though? 16:26:41 ... ICE Transport doesn't let you look at all the pairs... is this an insight problem into the state of things? 16:26:54 ... could you talk a bit more on what happens then? 16:27:26 Sameer: this is a first set of improvements - listing all candidate pairs would be another improvement 16:27:50 ... allowing to keep candidate alive is to allow to switch to a different pair (with another additional API) 16:28:00 Peter: in practice, the UA would continue sending checks on that candidate pair 16:28:24 TimP: but then what happens when it SHOULD remove it - e.g. because it's no longer working; you don't want to prevent THAT default 16:28:59 Sameer: Removal would be only for redundant pairs; failed candidates would "deleted" 16:29:14 JanIvar: that seems confusing - maybe it's a bikeshedding issue 16:29:49 Sameer: the UA can also send cancelable events that aren't systematically cancelable 16:31:17 Dom: I think it's primarily a bikeshedding issue - important to keep separate "deleted" and "removed", possibly with better names 16:31:25 Jared: the RFC uses "prune" 16:31:40 Sameer: my initial proposal had this; we could revert to that 16:31:52 ... I'm hearing most support for the cancelable approach, happy to write a PR for that 16:32:09 RESOLVED: Write a PR for a cancelable event approach 16:32:10 Topic: RtpTransport Follow-up 16:32:11 [slide 51] 16:32:20 RRSAgent, draft minutes 16:32:21 I have made the request to generate https://www.w3.org/2023/05/16-webrtc-minutes.html dom 16:33:00 s/51/50 16:33:03 [slide 51] 16:35:02 TimP: I'm not convinced these are use cases :) 16:35:10 Peter: fair - this are things that people want to do :) 16:35:13 [slide 52] 16:35:28 [slide 53] 16:35:41 [slide 54] 16:36:10 [slide 55] 16:36:34 [slide 56] 16:36:43 Peter: Depacketization would be built-in in the jitter buffer 16:36:59 [slide 57] 16:38:03 [slide 58] 16:38:10 [slide 59] 16:38:38 s|2023Apr/att-0001/WEBRTCWG-2023-04-18.pdf|2023May/att-0000/WEBRTCWG-2023-05-16.pdf 16:38:44 [slide 60] 16:39:10 RRSAgent, draft minutes 16:39:11 I have made the request to generate https://www.w3.org/2023/05/16-webrtc-minutes.html dom 16:39:35 Peter: what use cases would be needed to make this proposal more appealing? are there other type of examples people would like to see? 16:39:43 ... how much of the gap should be filled by us vs libraries? 16:39:47 ... would an explainer be useful? 16:39:58 Bernard: very helpful to see examples, thanks 16:40:19 ... how would SSRCs be handled? would that be another gap? 16:40:34 ... in a conference system, keeping track and routing ssrcs is a lot of work 16:40:45 ... would there be some help to manage this in the API? 16:41:31 ... Another question is about the depacketizer - would there be a generic video buffer distinct from the packetizer? 16:41:53 Peter: you're highlighting that an RTP demuxer would also be part of the gap 16:42:07 ... it is indeed in the gap, it could be provided 16:42:34 Bernard: it would provide an handler for a particular stream - that would enable handling them differently 16:42:54 Peter: for the depacketizer, audio and video need different handling since it's so trivial in audio 16:43:19 Bernard: how would audio/video sync work? I regularly get that question 16:43:27 Peter: worth adding to the gap indeed - thanks! 16:44:09 Jan-Ivar: looking at the use cases - it feels like we're abandoning the original WebRTC API with this, which worries me a bit 16:44:20 ... maybe we should solve time sync on the datachannel e.g. with timecodes 16:44:31 ... I'm worried that focusing on the new API means we don't solve it with the existing API 16:44:59 ... likewise, maybe we should look at exposing the same codecs as WebCodecs without having to use this low level API 16:45:28 ... This API seems to be using events (vs WhatWG Streams) and operating at very low level 16:45:43 ... I'm not sure all these use cases should or need to be solved with a new API 16:45:57 ... it feels this may be stepping on WebTransport (although of course WT isn't P2P) 16:46:19 Peter: WT is not P2P (yet), doesn't have real-time congestion control (yet), doesn't interoperate with RTP endpoints (ever) 16:47:11 ... in terms of WebCodecs - I designed an alternative with media senders / receivers for the media half of RTPSender/Receiver - it ended up being almost a duplicate of WebCodecs 16:47:42 ... in terms of abandoning the WebRTC API - that's part of this "filling the gap" approach I'm suggesting 16:48:02 ... overall, I wouldn't be sad if we end up abandoning RTCPeerConnection because something better emerges 16:48:34 ... there are a lot of things that we can continue to do incrementally, but I don't think we're going to get to all of it, esp at the pace we're going 16:49:04 Youenn: I've had people asking for similar features 16:49:30 ... for HEVC, there are some niche usages where ti would be hard to get UA support, and an RTPTransport would allow to fulfill these 16:50:00 ... what is the future though? would we still evolve RTCPeerConnection? it comes with some consistency and performance advantages 16:50:28 ... it would be interesting to see if you can get good performance with a WebCodecs / RTCTransport approach compared to RTCPC 16:50:28 ... An explainer would be good 16:50:42 ... Wrt packetizer, jitter buffer - WT apps would likely want something like that 16:51:03 ... we should check if a pure JS based approach isn't sufficient - this should be doable 16:51:13 ... we should focus on RTPTransport for P2P 16:51:38 ... it would still be useful to design packetizer API / jitter buffer API to help with protocol integration, but it should not be the focus 16:52:06 Harald: I do want to explode the PeerConnection object in smaller objects 16:52:40 ... but I would rather concentrate in defining APIs à la Transform API - it defines a dividing point to split the encoder from the packetizer 16:52:53 ... I'm not happy with the API we've proposed for this and am trying to reshape it 16:53:25 ... but I think we can go faster to where we want if we make it possible to plug your own object into an existing PeerConnection 16:53:34 ... breakout box did a good job at that 16:53:55 ... breaking things apart that way that leaves working systems, this will allow us to get there faster 16:54:33 Jared: RTPTransport makes me picture a nicely packaged component for SFUs 16:54:49 ... I would rather let people use this without having to fully unbundle everything 16:55:00 ... I'm worried by leaking API boundaries 16:55:23 ... like congestion window pushback where control congestion messages can end up telling the encoder to drop frames 16:55:31 ... how many of these leaky abstraction will we need to support? 16:55:58 Bernard: both mediacapture transform and encoded transform use WHATWG streams 16:56:12 ... Youenn had raised issues about that usage, not all of them have been put to bed 16:56:37 ... the danger of not using Streams is likely to create issues when using it in workers 16:56:59 ... transfering individual chunks or frames to worker isn't going to work very well 16:57:35 Youenn: FWIW, the Streams API issues are being addressed (for transferring ownership) 16:57:42 Bernard: but not for the one-object per pipeline 16:57:51 Youenn: might be the next thing worth addressing there indeed 16:58:05 Peter: I'm hearing support for an incremental approach towards something like RTPTransport 16:58:23 RRSAgent, draft minutes 16:58:24 I have made the request to generate https://www.w3.org/2023/05/16-webrtc-minutes.html dom 18:20:24 Zakim has left #webrtc