15:55:58 <RRSAgent> RRSAgent has joined #webrtc
15:56:02 <RRSAgent> logging to https://www.w3.org/2023/11/21-webrtc-irc
15:56:02 <Zakim> Zakim has joined #webrtc
15:56:05 <dom> Meeting: WebRTC November 2023 meeting
15:56:05 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/November_21_2023
15:56:08 <dom> RRSAgent, make log public
15:56:13 <dom> Chairs: HTA, Jan-Ivar, Bernard
16:03:03 <dom> Present+ Harald, Elad, Henrik, Riju, SunShin, Youenn, Dom, Florent, Guido, PatrickRockhull, PeterThatcher, Fippo, Sameer, Bernard, AlfredHeggestad, TovePetersson,
16:03:30 <dom> Recording is starting
16:04:45 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Nov/att-0005/Copy_of_WEBRTCWG-2023-11-21.pdf
16:05:03 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/ Mediacatpure-extensions
16:05:03 <dom> SubTopic: Issue #121: Background Blur: Unprocessed video should be mandatory
16:05:03 <dom> [slide 11]
16:05:23 <dom> Hta: related to "three thumbs up" that will be presented later in the agenda
16:05:30 <dom> Elad: that one will focus on the mute state
16:06:20 <dom> Hta: it would be much easier when there are multiple layers that can have an opinion on whether an effect will be applied if there was a way to disable the effect
16:06:28 <dom> Present+ PaulAdenot
16:06:42 <dom> Riju: this sounds nice, but not clear that can be supported on MacOS at the platform level
16:06:49 <dom> Present+ Jan-Ivar
16:06:53 <dom> Present+ TimPanton
16:07:10 <dom> Youenn: it's correct that there are OSes where this would not be possible at the moment
16:07:29 <dom> ... this "MUST" would not be implementable on these OSes
16:07:52 <dom> ... making it a SHOULD dependent on it being feasible (e.g. some cameras may not allow it)
16:08:47 <dom> HTA: we could have a MUST with a note that this won't be implemented everywhere
16:08:57 <dom> Youenn: the risk would be to discourage implementation of background blur
16:09:06 <dom> Present+ EeroHäkkinen
16:09:14 <dom> Youenn: SHOULD would be preferable
16:09:35 <dom> HTA: I'll try to phrase something making it clear it would only be if there is a good reason not to
16:10:04 <dom> Elad: background blur is a problem we want to solve, but one of several issues related to these effects and post-processings that can be applied by the UA or the OS
16:10:39 <dom> ... it's a complicated space, we shouldn't tie our hands too early; we could come back with a more generalized approach in December
16:10:58 <dom> HTA: next time I expect there will be a specific proposal to review
16:11:18 <dom> Bernard: for audio, we had a way to request specifically unprocessed audio
16:11:31 <dom> ... it helps in that it is forward compatible
16:12:22 <dom> Jan-Ivar: capabilities should reflect what can and cannot be done
16:12:31 <dom> ... not try to impose what the OS can do
16:12:46 <dom> HTA: we've helped push VP8 support at the OS level, so this wouldn't be unprecedented
16:13:05 <dom> TimP: if I had a system setting to enforce background blur, I wouldn't want the UA to override it
16:13:10 <dom> ... this has privacy impact
16:13:30 <dom> ... this also illustrates that there may be different rules across capabilities
16:14:09 <dom> HTA: discussion to continue on the bug with considerations on privacy, OS-implementability, and harmonization across effect/post-processing
16:14:14 <dom> SubTopic: Issue #129: [Audio-Stats] Disagreement about audio dropped counters
16:14:14 <dom> [slide 12]
16:14:48 <dom> Henrik: in previous meetings, we agreed on a set of audio stats to measure glitches
16:15:30 <dom> [slide 13]
16:17:59 <dom> Henrik: look for a decision on whether to expose audio frame drops to JS
16:18:13 <dom> [Fippo emoji-supports]
16:18:33 <dom> TimP: +1 on exposing it; there are ways to react too (e.g. turning up FEC, changing p-time, ...)
16:18:38 <dom> ... it's useful for the app to see this
16:18:54 <dom> youenn: it's fine as long as there is no fingerprint issues - we should look at that
16:19:09 <dom> ... e.g. maybe audio drop patterns may identify a device
16:19:16 <dom> ... we should have the same discussion with videoframe drops
16:19:37 <dom> ... on macos there is no way for video frames to be dropped
16:20:03 <dom> Paul: this is about local things - no network or packets involved
16:20:11 <dom> ... i.e. microphone to the page only
16:20:34 <dom> ... re fingerprinting, if there is a certain device showing issues, the UA is responsible for dealing with it
16:20:46 <dom> ... FF cannot drop audio between microphone and the Web page - it cannot happen
16:21:15 <dom> ... except if the machine is overloaded with real-time threads, but that's an edge case since at that point other threads wouldn't work either
16:21:43 <dom> ... so this feels like UA architectural bugs, and thus not something that should be exposed to the Web
16:22:17 <dom> ... for videos, dropping frames would be expected since the perceptual constraints are different
16:22:54 <dom> Henrik: any time we do a perf related experiment, we see issues
16:23:16 <dom> ... I don't think we can mandate that these issues can't happen
16:23:31 <dom> ... even if it was only a UA bug, there would still be value in exposing this for e.g. bug report
16:23:40 <dom> Harald: 3 parties in this game: the platform, the UA, the app
16:23:50 <dom> ... all 3 have opportunities to mess up audio
16:23:57 <dom> ... independently
16:24:15 <dom> ... many UAs are running on multiple OS, multiple versions of OS with different features and different bugs
16:24:25 <dom> ... there will be many cases of these combinations
16:24:31 <dom> ... Paul mentioned instrumentation and telemetry
16:24:48 <dom> ... the use case of WebRTC is so small that you have to have dedicated telemetry to make it show up at all
16:25:14 <dom> ... having the ability to have the application report on what happens when *it* runs, and not in the general case when the UA runs is important
16:25:26 <dom> ... my conclusion is that we should expose this to JS
16:25:56 <dom> TimP: is it really the case that changing the framerate and encoder settings would have no impact?
16:26:02 <dom> Paul: this is before encoding
16:26:16 <dom> Harald: this would have an impact in the PC
16:26:29 <dom> Henrik: adding encoder load could have an impact
16:27:01 <dom> TimP: surely the sample rate from the microphone is affected by p-time?
16:27:49 <dom> ... overall, it's not implausible you could influence it from the app; but in any case, would be good to have the information
16:28:00 <dom> youenn: native apps have access to this info
16:28:17 <dom> ... the app could decide e.g. mute capture in some circumstances
16:28:35 <dom> ... unless there are security or privacy issues, I don't see a reason not to expose it to Web apps as well
16:28:55 <dom> ... in terms of telemetry, the app could have telemetry of its own
16:29:03 <dom> ... I still support implementing it
16:29:30 <dom> Paul: there is nothing you can do in a Web page that will change the complexity of the CPU usage on the real-time thread that is used for input, except disabling @@@
16:30:21 <dom> Henrik: we have real-world data from Google with playout glitches due to lost frames; software/hardware encoder has an impact, also quality of device
16:31:15 <dom> HTA: I'm hearing rough consensus except for Paul
16:31:37 <dom> Jan-Ivar: I support Paul; FF would not implement this API since we don't think it's needed
16:32:12 <dom> Topic: WebRTC Grab Bag
16:32:12 <dom> Subtopic: -> https://github.com/w3c/webrtc-extensions/issues/146 Issue 146 Exposing decode errors/SW fallback as an event
16:32:12 <dom> [slide 14]
16:33:24 <dom> youenn: looking at the privacy comment, I'm not reading it as "this is fine"
16:33:33 <dom> Present+ Carine
16:33:59 <dom> youenn: I don't see how we could isolate these events across origins
16:34:14 <dom> ... we could try to go with a PR, but this will need a closer privacy look before merging
16:34:26 <dom> Present+ StefanHolmer
16:34:40 <dom> Subtopic: -> https://github.com/w3c/webrtc-svc/issues/92 Issue 92: Align exposing scalabilityMode with WebRTC “hardware capabilities” check
16:34:40 <dom> [slide 15]
16:35:06 <dom> [slide 16]
16:36:17 <dom> [slide 17]
16:37:01 <dom> Bernard: some of the modes are supported, but not as "smooth" or "powerEfficient" despite the machine being high specs
16:37:11 <dom> ... the hardware acceleration is not exposed for SVC
16:37:29 <dom> Henrik: in ChromeOS we can do some of the SVC modes in power efficient; L1T1 in Windows
16:37:45 <dom> ... there is what the device can do and what the UA has implemented, and how accurately this is represented in powerEfficient
16:38:08 <dom> ... on Windows lots of devices where L1T2 is available but not exposed in the UA yet
16:39:01 <dom> Bernard: webrtc-pc doesn't expose whether something is powerEfficient, only if it is supported
16:39:15 <dom> [slide 18]
16:39:29 <dom> RRSAgent, draft minutes
16:39:30 <RRSAgent> I have made the request to generate https://www.w3.org/2023/11/21-webrtc-minutes.html dom
16:40:12 <dom> Bernard: proposal is to bring something back to SVC if/when media capabilities limits this exposure
16:40:36 <dom> Jan-Ivar: hardware support being limited today doesn't mean it will be tomorrow
16:40:46 <dom> ... but in general, +1 to bringing this to media capabilities
16:41:09 <dom> ... the "capture check" is not necessarily a good fit for all use cases (e.g. games over data channel)
16:41:26 <dom> ... it also risks driving to escalating permissions
16:42:12 <dom> Henrik: I'm hearing agreement that media capabilities should solve this
16:43:15 <dom> Topic: -> SDP https://github.com/w3c/webrtc-encoded-transform/issues/186 Issue 186: New API for SDP negotiation
16:43:15 <dom> [slide 24]
16:44:10 <dom> HTA: same functionality, different API shape
16:44:12 <dom> [slide 25]
16:46:08 <dom> [slide 26]
16:47:08 <dom> [slide 27]
16:47:36 <dom> HTA: having the API on the transform, the app would have to talk to the transform, and the transform have to talk to the SDP which seems bad
16:47:56 <dom> Jan-Ivar: the Transform between the encoder and the packetizer is not the one we're talking about
16:48:31 <dom> ... we're talking about RTCScriptTransform which runs on the main thread and which would say "here is how this transform should be exposed as in terms of codecs"
16:48:40 <dom> ... it's an inherent property of the Transform
16:49:40 <dom> ... I don't think we should organize APIs around SDP but around the functionalities as perceived by Web developers
16:50:01 <dom> HTA: the purpose of isolating SDP is to not entangle SDP with stuff that we might want to use without SDP
16:50:14 <dom> ... keeping a distinction is a good thing in that sense
16:50:36 <dom> Jan-Ivar: transceiver.sender/receiver are always created at the same time
16:50:42 <dom> HTA: at the moment yes
16:51:18 <dom> Youenn: at some point we'll have SFrame packetization that we'll likely want to use in connection with SFrameTransform
16:51:57 <dom> ... I wonder if looking how SFrame would work in this model would help establish the underlying infrastrcture
16:52:14 <dom> HTA: when installing an SFrameTransform, SDP has to be affected, but the sframe spec doesn't say how yet
16:52:43 <dom> Bernard: the SFrame packetization spec covers RTP
16:52:54 <dom> Peter: it's a work in a progress
16:53:22 <dom> HTA: not sure we can depend on that progress to drive our architectural decisions
16:53:47 <dom> Henrik: in my mind, a transceiver and an m-section map one-to-one
16:54:25 <dom> ... in Jan-Ivar's proposal where the Transform contains information about SDP - would the only difference be that the transceiver ask the Transform what is its payload type?
16:55:04 <dom> ... does it make a huge difference between the two? e.g. will it be the same number of methods
16:55:26 <dom> Jan-Ivar: my proposed API is much simpler - it's one step done through the constructor of the transform
16:57:02 <dom> ... it's not true that Transceiver encompasses all the negotiation needs (e.g. addTrack / createDataChannel)
16:57:57 <dom> ... using two codec names is confusing - the packetization should be a subproperty of the codec
16:58:07 <dom> HTA: I look forward to a more flushed out proposal in that direction
16:58:12 <dom> [slide 28]
16:59:41 <dom> Fippo: I would say the PT
16:59:59 <dom> ... since the mime type is the result of the lookup of the payload type
17:00:11 <dom> Jan-Ivar: I was going to say the opposite, but Fippo's point makes sense
17:00:19 <dom> ... how would you find the PT?
17:00:32 <dom> Harald: go through the codecs from getParameters
17:01:10 <dom> Henrik: if if it's one-to-one mapping, the ergonomics would go for mime type
17:01:34 <dom> HTA: if we move a frame between PC, the PT may not have the same meaning, when the mime type does
17:02:12 <dom> youenn: I would go with PT; PT and mime going out of sync may suggest there should be a different field for packetization
17:03:01 <dom> HTA: you could specific that once you enqueued a frame, it sets the other based on the other and ignores it if it can't find the mapping
17:03:14 <dom> [fippo: +1]
17:03:36 <dom> HTA: I'm hearing slightly favorable to mime type, but this needs more discussion - will summarize it on the github issue
17:03:45 <dom> Topic: -> https://github.com/w3c/webrtc-rtptransport RtpTransport
17:03:46 <dom> [slide 31]
17:04:09 <dom> [slide 33]
17:04:53 <dom> Subtopic: Bandwidth Estimation
17:04:54 <dom> [slide 35]
17:05:39 <dom> [slide 36]
17:06:26 <dom> Bernard: the RtpTransport is both for sending and receiving - right?
17:06:29 <dom> Peter: right
17:07:01 <dom> Stefan: have you thought about what it would look like for a user of this API that would want to implement its own bandwidth control in the Web app?
17:07:09 <dom> ... would this done through this API or something else?
17:07:24 <dom> Peter: I have thought about that, think we should discuss it, but not today :)
17:07:55 <dom> Jan-Ivar: the other transports have back-pointers from the transport; shouldn't this be under the dtlsTransport
17:08:26 <dom> Peter: the dltsTransport should be under the rtpTransport, but we can't change this at this point
17:08:58 <dom> Orphis: with Bundle semantics, I think you really need it on each media section
17:09:09 <dom> Subtopic: Forwarding
17:09:11 <dom> [slide 38]
17:09:54 <dom> Subtopic: Using existing m-lines
17:09:56 <dom> [slide 40]
17:10:25 <dom> [slide 41]
17:12:00 <dom> [slide 42]
17:12:57 <dom> HTA: this crosses multiplexing
17:13:33 <dom> ... I would rather not make it too easy to go over these bumper lanes
17:14:19 <dom> Jan-Ivar: we're already way too low level than I'm comfortable with; this is a very low level API
17:14:31 <dom> ... if every sender has an RTPTransport, does it also show up if it has a track
17:14:41 <dom> ... I was imagining more of a new type of datachannel
17:15:05 <dom> ... this would mutually exclusive with the track API
17:15:37 <dom> ... rather than exposing a JS API to programatically send packets
17:16:00 <dom> ... the benefits of using a writable we're seeting for WebTransport helps let the UA manage the throughput and keep performance
17:16:33 <dom> ... I'm looking for an API that allows to change the source of RTP rather than control of RTP
17:16:57 <dom> Florent: how would the API work with the various ways of encrypting data in an RTP packet, e;g. cryptex
17:17:10 <dom> s/e;/e.
17:17:46 <dom> Peter: we haven't talked yet about encryption of header extensions - a good topic to cover, like Stefan's; this would need more time
17:18:54 <dom> Henrik: you could register to send for a specific payload
17:19:15 <dom> Peter: I'd need to see examples of what you're thinking to help
17:19:51 <dom> ... I think a more in-depth design meeting would be useful
17:20:31 <dom> Bernard: the point about cryptex makes it clear that there needs to be a facility to create a fully-formed packet for the wire
17:21:45 <dom> ... WhatWG streams won't do SRTP when you call write
17:25:07 <dom> Dom: so the Chairs should figure a separate meeting for more in-depth design discussions
17:26:05 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/issues/39 Multi-mute (Three Thumbs Up - Setup)
17:26:05 <dom> [slide 58]
17:27:52 <dom> Elad: we'll focus on muting today
17:27:56 <dom> [slide 59]
17:28:03 <dom> RRSAgent, draft minutes
17:28:04 <RRSAgent> I have made the request to generate https://www.w3.org/2023/11/21-webrtc-minutes.html dom
17:28:47 <dom> [slide 60]
17:29:07 <dom> [slide 61]
17:30:05 <dom> [slide 62]
17:30:12 <dom> Elad: this is a problem worth solving
17:30:18 <dom> [slide 63]
17:32:01 <dom> [slide 64]
17:32:36 <dom> Elad: I think exposing upstream state is more important than changing it
17:33:03 <dom> [slide 65]
17:33:53 <dom> Elad: the mute event doesn't suffice - it is defined as a temporary inability to provide media
17:34:14 <dom> ... muted refers to the input to the mediastreamtrack
17:34:20 <dom> [slide 66]
17:35:10 <dom> [slide 67]
17:35:39 <dom> [slide 68]
17:37:22 <dom> [slide 69]
17:37:59 <dom> [slide 70]
17:39:38 <dom> jan-ivar: hear 3 proposals: muteReasons, potentiallyActionable, requestUnmute
17:39:42 <dom> ... I support requestUnmute
17:40:21 <dom> ... I think muteReasons should only have 2 states; things outside of the browser can be correlated across origins
17:41:26 <dom> ... regarding requestUnmute, we already have a toggleMicrophone in the media session API
17:42:28 <dom> Elad: re muteReasons, having more granularity would be nice to deal with the diversity of OSes
17:43:39 <dom> Jan-Ivar: the UA would be responsible to deal with the upstream OS when it can't deal with requestUnmute on its own
17:44:06 <dom> Youenn: I see convergence on requestUnmute
17:44:28 <dom> ... depending on how scary calling that method would be for the Web app, it may impact how much information to expose in muteReasons
17:44:49 <dom> ... in terms of the boolean, some of the definitions wouldn't be implementable e.g. in iOS
17:45:02 <dom> ... hence why we should focus on requestUnmute first
17:45:17 <dom> ... requestMute would be nice for later
17:45:32 <dom> Elad: requestMute would risk muting other apps - but it feels orthogonal anyway
17:45:56 <dom> ... re boolean, see [slide 66]
17:46:26 <dom> ... if we don't expose the distinction between UA and OS (e.g. call it "upstream"), would that work for you?
17:46:40 <dom> Youenn: I would want to better understand requestUnmute
17:46:56 <dom> ... I believe that will help drive the discussion on the boolean value - I'm not opposed to it
17:48:02 <dom> Elad: I would argue that the MuteSource is useful even if requestUnmute is never called
17:51:12 <dom> Youenn: the case that is interesting is the one where the user muted
17:51:43 <dom> Guido: right now, the spec doesn't define muted the way Youenn suggests; any interruption in the capture cause a muted event
17:51:57 <dom> ... it doesn't reflect user intent
17:52:57 <dom> Jan-Ivar: the examples from the spec refer to user-intended actions
17:54:09 <dom> ... maybe we should fix the spec to allow the distinction between a "temporal" mute and a user-intented mute
17:54:53 <dom> Elad: changing the spec and risking to break existing implementations will be painful compared to just adding a boolean
17:55:39 <dom> Jan-Ivar: I would be happy to propose slides with toogleMic / toggleCamera
17:55:49 <dom> ... mutesource has value (but not without so many values)
17:57:16 <dom> Harald: OS capabilites change over time, we shouldn't limit ourselves to these current capabilities
17:57:47 <dom> Guido: re media session, would this be re-using the event or interacting with the media session API?
17:58:52 <dom> Jan-Ivar: the event
18:00:05 <dom> Bernard: given how much content we didn't cover, should we schedule another meeting in December?
18:00:28 <dom> RRSAgent, draft minutes
18:00:30 <RRSAgent> I have made the request to generate https://www.w3.org/2023/11/21-webrtc-minutes.html dom