09:25:55 RRSAgent has joined #webrtc 09:25:59 logging to https://www.w3.org/2023/09/12-webrtc-irc 09:26:04 Meeting: WebRTC TPAC 2023 meeting 09:26:05 Agenda: https://www.w3.org/2011/04/webrtc/wiki/September_12_2023 09:26:05 Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Sep/att-0010/WEBRTCWG-2023-09-12.pdf 09:26:05 Chairs: HTA, Jan-Ivar, Bernard 09:31:42 Present+ DomHM, HTA, Bernard, Fippo, Guido, Florent, TonyHerre, SunShin, Tove, ZhangShishen, PeterThatcher 09:32:47 Present+ Youenn 09:33:08 tetter has joined #webrtc 09:33:54 Present+ ShridharMajali, Sameer, Varun 09:34:19 Present+ JIB 09:36:16 Present+ Carine, aheggestad 09:36:29 Recording is starting 09:37:10 Peter_Thatcher has joined #webrtc 09:37:52 youenn has joined #webrtc 09:37:53 jesup has joined #webrtc 09:38:11 vr000m has joined #webrtc 09:39:09 Bernard has joined #webrtc 09:39:31 scribe+ dom__, youennf 09:39:53 herre has joined #webrtc 09:40:06 Present+ ChengChen 09:40:22 [reviewing WebRTC related meetings and breakout sessions during TPAC] 09:41:36 Present+ Youenn, Riju, PeterB, RyosukeNiwa 09:44:20 Topic: State of the WebRTC WG 09:44:20 [slide 13] 09:45:28 [slide 14] 09:46:45 hta: I've looked at which repos have had what activity since last TPAC 09:46:45 caribou has changed the topic to: TPAC Meeting in progress - https://www.w3.org/2011/04/webrtc/wiki/September_12_2023 09:46:45 ... mediacapture-main still progressing towards Rec, incl removing unimplemented features, e.g. permissions query 09:46:52 ... mediacapture-extensions has been active as a holding pens for new ideas 09:47:00 ... webrtc-pc fixing bugs and merging some extensions, using the mechanism to keep track of diffs from the Rec 09:47:09 ... somewhat painful, but it works 09:47:26 ... stats behaving like a living standards - adding and removing stuff 09:47:38 ... lots of discussions on use cases 09:48:19 ... webrtc-encoded-transform, looking at new functionalities 09:48:19 ... webrtc-ice - may need to be killed off now that the work is migrating in webrtc-extensions 09:48:37 ... platform processing needs synchronization with the Media WG, there were discussions yesterday about video filters 09:49:01 ... Screen Capture - largely pursued in SCCG with whom we meet on Thu 09:49:26 ... The remaining docs are mostly stable - they're implemented but haven't seen much change 09:49:43 ... some because no change is needed, some because noone is driving the changes 09:50:22 Youenn: re mediacapture-main being close to Rec, it still have 30 open issues 09:50:37 ... should they be triaged, moving some to -extensions 09:51:54 riju has joined #webrtc 09:52:27 ... or focus the group's effort on closing it 09:52:41 Dom: +1 on triaging issues - JIB is looking at it AFAIK 09:53:04 ... on the unattended repos, some have pending requests from the community; I would hope we get more attention/ownership on them 09:53:47 JIB: re mediacapture-main, I don't think there are any big remaining open issue - please chime in if you feel differently 09:54:21 HTA: one of the sticky issue is around localization of device names; discussion between TC39 and the I18N WG have made significant progress on establishing a JS pattern for this 09:54:28 ... I personally feel that lack of progress means we shouldn't block on this 09:55:17 Topic: -> https://github.com/w3c/webrtc-nv-use-cases/ WebRTC Extended Use Cases 09:55:17 Subtopic: Relationship between use case docs and other resources 09:55:17 [slide 19] 09:55:19 [slide 20] 09:56:05 (no objection expressed on the proposal) 09:56:07 [slide 21] 09:56:57 (no objection expressed on the proposal) 09:57:06 [slide 24] 09:57:08 s/21/22/ 09:58:21 HTA: if there are links from use cases to API proposals, they need to be updated each time new proposals emerge 09:58:59 ... I would rather say that use cases should usually not link to API proposals 09:59:07 (no objection expressed on the amended proposal) 09:59:23 Subtopic: Funny Hats use case 09:59:23 [slide 26] 10:02:07 hta: e.g. if background blur can happen at different layers, frames need to be annotated with the fact they've been processed - which means deviating from the standards frame format 10:02:24 ... which needs to be surfaced in SDP as a non-standard codec 10:03:11 Bernard: we need to add matching requirements 10:03:19 HTA: I can make a PR towards that 10:03:28 Bernard: there is also a metadata related to this topic 10:04:35 HTA: I wanted confirmation the WG recognizes these usages as relevant 10:05:33 To help this use case, being more precise about the potential type of metadata/application might be more convincing. 10:05:50 Riju: re background blur happening at different layers, … 10:06:04 henrik: re metadata - are we thinking app-specific metadata or standardized ones? 10:06:29 hta: so far, we've seen app-specific metadata, and I don't see them going away even if there are standardized ones 10:06:35 henrik: +1 10:06:42 s/To help/youenn: To help 10:07:01 hta: they need to be standardized when we need interoperability 10:08:54 JIB: SDP negotiation seems to assume we need to modify metadata in frames - another approach might be to bring media sync in data channels 10:08:54 ... adding details on when in-frame metadata is needed 10:08:54 henrik: the problem with datachannel is there is no guarantee of having received it before the frame 10:08:54 ... plus the need to timestamp the info 10:08:57 RESOLVE: HTA to get a PR to articulate the requirements for this use case 10:09:04 s/VE/VED 10:09:25 Subtopic: Low Latency Streaming: Summary 10:09:28 [slide 27] 10:09:43 Subtopic: Low Latency Streaming: Game Streaming use case 10:09:44 [slide 28] 10:10:16 [slide 29] 10:11:06 Bernard: issue #80 - gaming has e.g. spatial audio; they need raw audio data to manage this 10:11:44 ... also related discussions to WASM codecs 10:11:50 [slide 30] 10:11:51 [slide 31] 10:12:19 Bernard: do we need to add related requirements? L16 is popular and important for spatial audio in gaming 10:13:09 HTA: what is raw audio data? we should be specific if we mean L16 10:13:28 ... "access to" is unclear on whether that's at the sender or receiver side, which impacts what goes across the wire 10:13:51 Bernard: in the original issue, it's more on the sender side (although the receiver will need to know about the codec) 10:14:03 HTA: which then intersects with the SDP munging requirement 10:14:32 Youenn: I think the real issue is about pluggable audio codecs (rather than raw audio data) 10:14:57 ... Access to raw audio data is already provided by the platform 10:15:21 Henrik: how is this different from WebCodecs? 10:15:46 ... is this distinct from our discussion of plugging WebCodecs in RTCPC? 10:16:16 Youenn: it's specific to audio; WebCodecs is likely one of the solutions or part of it 10:16:27 HTA: should we rename this to pluggable audio codec? 10:16:48 Bernard: does this issue need to be addressed for the low latency use case? or does it go somewhere else? 10:17:22 HTA: it's not linked to specifically to low-latency use case; it has a requirement for pluggable audio codec, but not for this particular use case 10:17:57 Youenn: cloud gaming is behind this use case, but it's not about low latency streaming 10:18:21 ... not sure if this is about non-standard codec, or codec not supported by the platform 10:19:10 Peter: +1 to Henrik - the desire to use a different (possibly a custom/proprietary) one is not streaming specific, it's broader 10:19:19 Bernard: do we need a new use case for this? 10:19:24 Peter: if we don't have one, we should 10:19:57 Youenn: this is linked to wanting it on RTP (rather than via a datachannel) 10:20:26 Peter: data channels for audio is problematic due to e.g. congestion control, or compatibility with existing RTP endpoints 10:20:43 Youenn: but the latter conflicts with non-standards codecs 10:20:55 HTA: although not with codecs not supported by browsers 10:21:23 igarashi_ has joined #webrtc 10:21:40 RESOLVED: detach #80 from low-latency use case; create a new use case related to pluggable audio codecs 10:21:56 Bernard: I can take care of defining the use case 10:22:19 Peter: is this specific to audio? or should that encompass video? 10:22:23 igarashi_ has joined #webrtc 10:22:28 Bernard: the OP was specific to audio 10:23:08 ... the question exists in video for HEVC - but maybe we can assume it will be natively supported in WebRTC 10:23:27 HTA: a major difference between audio & video is the integration with breakout box 10:24:19 Present+ ErikSprang, RandellJesup 10:25:16 [slide 33] 10:26:44 Harald: we should write an explainer that explains that jitterBufferTarget satisfies N38 10:27:04 RESOLVED: add a note in webrtc-extensions that jitterBufferTarget satisifes req N38 10:27:29 SunShin: presenting PR #118 to clarify game streaming requirements 10:29:50 [slide 34] 10:29:57 [slide 35] 10:30:48 Requirements make sense. Layered codecs for example may help with recovery and consistent latency at cost of resolution changes. UAs are allowed to display corrupt frames today instead of pausing, but there's no way to ask for it 10:33:26 SunShin: these 4 four requirements would help improve the current experience of game streaming 10:33:33 [slide 36] 10:34:08 Bernard: RFC8884 already recommends RPSI - the IETF requires it for the stack 10:34:21 ... the issue is not whether to implement, but there are practical issues 10:34:23 [slide 37] 10:34:54 ... RPSI support in HEVC was mentioned in discussion on HEVC in WebRTC 10:35:22 ... support for RPSI with VP8/9 was from removed from libwebrtc in 2017 10:35:51 ... if we can figure a way to address this in the code base, the W3C requirements would not be an obstacle 10:36:13 ... since it's already recommended by the IETF, I'm not sure we need a requirement for it 10:36:30 Youenn: is there a case where a WebApp would want to enable or disable RPSI, or can that be left to the UA? 10:37:08 SunShin: we would want app-control for RPSI 10:37:23 Bernard: this can be negotiated with SDP munging (if it was in WebRTC) 10:37:45 Youenn: but SDP munging isn't great 10:39:03 ... if we think it needs to be app-exposed, then having the requirement explicit is good, so that it gets exposed via an API 10:39:37 Bernard: I don't recall a stack with app control on RPSI; it's typically something the RTP stack just does it 10:39:37 ... that's how it was originally implemented for VP8/VP9 10:40:21 Elad: @@@ 10:40:40 ... we would want to switch between different modes 10:41:15 HTA: there was a Google decision to switch from RPSI to Loss Notification - any recollection about this? 10:41:38 HTA: looks like we need more information 10:41:47 ... some of it needs to go back to IETF 10:43:08 RESOLVED: re N48 & N49, investigate what information needs to be passed on the wire and figure if it's IETF or API work to satisfy this requirement 10:43:25 Bernard: this will come out of the HEVC implementation in WebRTC from which it should bubble up 10:44:29 SunShin: re N50 - we still need a transport wide RTCP 10:45:28 Andreas has joined #webrtc 10:45:42 @@@: there should be implementations for that 10:45:51 s/@@@/Erik Stags/ 10:45:59 Bernard: we're running out of time, maybe finish discussion in overflow session 10:46:09 HTA: 38-43 for overflow 10:46:26 Present+ Palak 10:46:31 Topic: -> https://github.com/w3c/webrtc-encoded-transform/ Modifications for Low Latency Fanout (WebRTC Encoded Transform) 10:46:32 [slide 46] 10:47:30 [slide 47] 10:47:58 [slide 48] 10:48:50 [slide 49] 10:49:04 [slide 50] 10:50:23 HTA: platform objects with serialization and de-serialization don't require the output be JS observable 10:51:12 Peter: you're talking about encoded frames 10:51:15 Palak: yes 10:51:30 Peter: would a constructor for encoded frames sufficient? would it alleviate some of these needs? or is it unrelated? 10:51:49 HTA: we discussed constructors; for this particular use case, defining the constructor would require defining all the metadata 10:52:13 ... whereas only modifying the needed metadata is a significant simplication 10:52:25 ... constructors are needed for other use cases 10:52:51 Peter: re forwarding encoded frame to another PC - how do you know the available bandwidth to know what you're capable of forwarding or not 10:53:22 HTA: the use case we're considering is for a PC on the same network where we assume similar bandwidth characteristics 10:53:41 Henrik: when forwarding or sending a frame, the question of having enough bandwidth is unrelated 10:54:15 JIB: I have a concerned on how we got here; WebRTC encoded transform is taking a pipe out of the wall and plug JS to allow to transform incoming frames 10:55:02 ... this has become a very powerful opportunity to bring more changes 10:55:02 ... it changes the ability for UA to make optimal decisions based on understanding what's happening 10:55:02 ... I wonder if we need a different / more specialized API surface 10:55:35 ... re structure cloning, encoded frames don't come with different ownership (e.g. GPU) so I don't see an issue 10:55:56 Youenn: similar concerns in terms of use cases 10:56:51 ... bandwidth estimation may get fooled by having JS sending much bigger or lower data than the UA estimated 10:57:44 ... a different API shape would be preferable, with making the JS being responsible for encoding, telling the UA clearly needs to approach bandwidth estimation differently 10:58:15 ... not sure what mechanism would be used in case of packet loss 10:58:27 ... and a Web app can't detect whether the network is susceptible to packet loss 10:59:13 ... an API where the Web App is repsonsible for encoding and is notified when something gets lost 11:00:10 HTA: we know this can be done because we have done it 11:00:33 ... there are alternative API shapes; I would try to pursue a few ones, but it hasn't been done 11:00:46 ... this particular one works and will be part of a solution in any case 11:02:26 Youenn: I think a dedicated API would be better 11:03:40 HTA: I think webrtc-encoded-transform is the right document; I wouldn't want to wait for the perfect solution to arrive 11:03:40 Youenn: this isn't a transform - this is about sending 11:04:24 HTA: new API shapes haven't be conclusive, and using the existing API has proved workable 11:04:46 Present+ LouayBassbouss 11:05:27 Youenn: we still don't have a solution for the metadata issue with encoded transform 11:06:20 RRSAgent, draft minutes 11:06:22 I have made the request to generate https://www.w3.org/2023/09/12-webrtc-minutes.html dom__ 11:06:33 RRSAgent, make log public 12:01:01 tetter has joined #webrtc 12:02:52 Topic: -> https://github.com/w3c/mediacapture-extensions/ WebRTC & Media Capture 12:05:51 youenn has joined #webrtc 12:06:18 scribe: youenn 12:06:44 slide 34: track stats API shape 12:07:17 Henrik summarising the issue 12:09:54 anssik has joined #webrtc 12:10:28 youenn: I would prefer that you would only start counting if there's a call. that's not the case since it's a getter 12:10:43 scribe+ caribou 12:12:12 present+ PaulAdenot 12:12:59 youenn: we encourage developers to do more on the main thread with that kind of API 12:13:26 PaulA: is this reimplementing stable state? 12:13:56 ... that's efficient, nice 12:14:01 Paul: stable state attribute in the HTML spec can be used for that. 12:15:12 HTA: update the counters whenever in stable state. A cache is fine to remove the overhead if you do not read. 12:17:21 Henrik: how about I modified this PR or do a follow-up to handle stable state. 12:18:16 Jan-Ivar: naming might be a remaining issue (videoStats vs. stats) but synchronous is fine. 12:18:54 Follow-up issues to handle are whether using stable state and whether renaming. 12:19:03 Issue 137 12:19:16 slide 51 12:21:31 Jan Ivar explaining the issue 12:24:30 youenn: how about extending the possibility for enumerateDevices to expose output device a little bit longer. 12:27:39 like done in Firefox for prompting cameras and microphones. 12:28:46 HTA: If devicechange event is fired, app will know AirPods are gone. If reload happens before, it will not know. 12:30:22 HTA: one time courtesy is not great. How about returning NotFoundError. 12:30:44 JI: Maybe a SHOULD would be good enough to give enough leeway for User Agents. 12:31:06 Issue 2899 12:34:41 present+ TimPanton 12:36:04 Peter: like proposal A. Proposal C might be to add a method getIceTransport(). 12:36:49 Henrik: is it useful to have sctp transport? 12:36:58 Jan Ivar: yes (see code in slide 53) 12:39:20 Florent: like proposal A but not make maxMessageSize nullable. 12:40:32 HTA: I like option C. For dtls transport, web pages had to do extra code to get it. 12:40:55 Jan Ivar: during initial negotiation, it is not clear which transport we will expose. 12:42:05 HTA: with maxBundle, you will get one transport. And you can compare the transport objects if you have multiple. 12:42:05 HTA: Proposal C (without doing proposal A). 12:44:05 Rough consensus to go with proposal A in the room. 12:44:19 TOPIC: ICE Controller API 12:45:30 Issue 171 12:50:44 Slide 62 12:50:45 jesup_ has joined #webrtc 12:53:45 HTA: why ArrayBuffer for transactionId? 12:53:54 Peter: transactionId is 20 bytes. 12:58:02 youenn: is it ok in a window? should it be in a worker? 12:58:12 Peter: nice to have in worker as well 12:59:51 consensus to move on with a PR for that API. 13:00:36 TOPIC: RTPTransport 13:04:46 Peter: new createRtpTransport() 13:05:08 Youenn: can you mix legacy senders/receivers with RtpTransport? 13:05:15 Peter: Yes 13:05:22 it should be feasible 13:05:39 Stefan: what about overreaching bwe? 13:05:51 Peter: packets would be queued and/or dropped. 13:07:57 Jan Ivar: what about creating several rtp transports? 13:08:13 Peter: will cover SDP negotiation later 13:08:29 slide 71 13:17:35 Randell: should be discussed in this WG. Incremental API is better. Worker-only. 13:18:09 Jan Ivar: similar to Randell, worker-only is good. 13:18:29 ... examples with WebRTC encoded transform would be good. 13:19:41 HTA: this is interesting. Why not having a RTPTransport getter on the peer connection? 13:20:19 Peter: tied to what is negotiated. Maybe this is enough. 13:21:03 ... trying to avoid the situation where packets sent is not what was negotiated by SDP 13:21:12 Bernard: Good to have workers. 13:22:07 Maybe good to separate them. 13:23:40 Peter_Thatcher has joined #webrtc 13:24:14 youenn: workers is good. 13:25:16 riju has joined #webrtc 13:25:16 I would concentrate on webcodecs + RTPTransport. Interaction with WebRTC encoded transform can come later. 13:25:16 youenn: having discussions at frame vs. packet level APIs would be good to come to a conclusion. 13:25:32 TimPanton: would be nice to be able to demultiplex processing. 13:28:00 HTA: limitation if there is a need for a new SDP m-line to talk with other peers. Maybe it is not that big a problem nowadays since apps tend to control both sides. 13:31:13 Jan Ivar: might need to think about back pressure (events vs streams) 13:32:38 HTA: sense of the room is that there is interest. Need to sort out the thread story (main thread/non main thread). Whether interop or not (SDP changes or not). 13:33:06 Consensus to get this work done here. 13:33:46 Should probably be a new document 13:34:53 TOPIC: SDP negotiation 13:52:19 dontcallmedom has left #webrtc 13:59:16 Summary: PT to MimeType -> arguments on both sides. Setting packetisers is also accepted but needs further discussion. 14:00:09 TOPIC: Back to WebRTC use case issues 14:08:55 vr000m_ has joined #webrtc 14:09:48 The Ultra low latency broadcast with fanout seems to be Application Layer Multicast, ALMs. There are quite a few papers and implementations with RTP. 14:15:38 youenn has joined #webrtc 14:15:40 Next step for PR123: continue discussion on GitHub with the intent to merge. 14:15:58 -> https://github.com/w3c/webrtc-nv-use-cases/pull/120 PR 123 14:16:27 TOPIC: Wrap-up and next steps 14:17:14 I have made the request to generate https://www.w3.org/2023/09/12-webrtc-minutes.html caribou 14:17:27 Transport draft could be a candidate for a new document. 14:18:39 HTA: we have more meetings this week 14:19:53 breakout on WebRTC UCs tomorrow as well 14:21:08 [end] 14:21:12 I have made the request to generate https://www.w3.org/2023/09/12-webrtc-minutes.html caribou 14:43:07 Zakim has left #webrtc 14:53:44 s/RyosukeNIwa// 14:53:51 s/PeterB/HenrikB/ 14:53:55 I have made the request to generate https://www.w3.org/2023/09/12-webrtc-minutes.html caribou 15:03:19 jesup has left #webrtc