14:48:57 <RRSAgent> RRSAgent has joined #webrtc
14:49:01 <RRSAgent> logging to https://www.w3.org/2024/06/18-webrtc-irc
14:49:01 <Zakim> Zakim has joined #webrtc
14:49:01 <dom> RRSAgent, make log public
14:49:41 <dom> Meeting: WebRTC June 18 2024 meeting
14:49:42 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/June_18_2024
14:49:42 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2024Jun/att-0002/WEBRTCWG-2024-06-18.pdf
14:49:42 <dom> Chairs: HTA, Jan-Ivar, Bernard
15:00:18 <dom> Present+ Bernard, Youenn, SunShin, Dom
15:00:48 <dom> Present+ JianjunZhu
15:01:36 <dom> Present+ Harald, Sameer, Jan-Ivar, TimP
15:02:55 <dom> Recording is starting
15:03:43 <dom> Present+ PatrickRockhill
15:03:51 <dom> Topic: Future meetings
15:03:57 <dom> Present+ TonyHerre
15:04:16 <dom> Bernard: there has been discussion of canceling our scheduled meeting on July 16 and instead meet on August 27
15:04:24 <dom> ... any objection to that plan?
15:04:29 <dom> [none expressed]
15:04:38 <dom> REsolved: cancel July 16 meeting and schedule one on Aug 27
15:05:58 <dom> Present+ FlorentCastelli
15:06:19 <dom> Topic: -> https://github.com/w3c/webrtc-charter/pull/83 Proposal: merging more extensions in WebRTC Rec
15:06:19 <dom> [slide 10]
15:09:44 <dom> Dom: any comment or objection to the proposal?
15:10:09 <dom> Jan-Ivar: agree on the extension spec being confusing
15:11:01 <dom> ... this isn't a charter change, right?
15:11:08 <dom> Dom: correct
15:14:22 <dom> Jan-Ivar: no objection, wonder if this could be used to mediacapture-main pre-Rec
15:14:30 <dom> Dom: yes, the process would allow for that
15:14:52 <dom> Bernard: what does reasonable test coverage encompass?
15:17:30 <dom> Dom: Basically enough to have confidence the implementation reasonably matches what we've specified
15:17:48 <dom> TimP: should we consider non-browser implementations in this context?
15:21:08 <dom> Dom: That may be something to consider independently of this policy; but my personal sense is that this WG should focus on browser interop, even though other implementation contexts are a great secondary impact of this work
15:22:47 <dom> Jan-Ivar: +1
15:23:27 <dom> Youenn: what constitutes a commitment? in some cases, it may be we're willing to implement but with unclear or a moving implementation roadmap
15:23:46 <dom> Dom: commitment could be a bug tracking link, a standards position from an implementor
15:24:04 <dom> RESOLVED: proceed with adopting new merge-guide guidance
15:25:13 <dom> Topic: -> https://lists.w3.org/Archives/Public/public-webrtc/2024May/0031.html WebRTC Charter Renewal
15:25:13 <dom> [slide 11]
15:25:26 <dom> Dom: any objection to proceeding with that plan?
15:26:08 <dom> RESOLVED: Proceed with editorial renewal of the WebRTC WG charter
15:26:14 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/ Media Capture Extensions
15:26:14 <dom> Subtopic: Issue #145 Consider adding onVoiceActivity event on MediaStreamTrack for audio
15:26:14 <dom> [slide 15]
15:28:11 <dom> Jianjun: is there support for adding such an event?
15:28:47 <dom> Youenn: the "unmute microphone" hint is a great use case; the media session API will help to fully mute the microphone, so helping to expose that state to the end user is a great idea
15:29:13 <dom> ... there is OS support for it already; so this is a valid use case
15:29:43 <dom> ... I'm not sure we can limit processing to only when the user is actively speaking since it may be too late to do the processing once the event has been received
15:30:54 <dom> Jan-Ivar: the two use cases may lead to different solutoins; I very strongly support the 1st use case, the 2nd may have privacy implications
15:31:02 <dom> s/oins/ions/
15:31:19 <dom> ... I'm not even sure you need a constraint for that - maybe if it's onerous for implementations to do that
15:32:41 <dom> Youenn: there might be use cases via audio stats, e.g. there is an rtp header extension to expose voice activity as a boolean, typically set by the encoder
15:33:10 <dom> ... providing that in real-time may make sense, but not sure if the event is the right approach, this will need more experimenting
15:33:46 <dom> Bernard: in terms of privacy implications, determining start/end of speech occurrences may reveal more information to the app than the user would expect
15:34:18 <dom> ... we could fire only "start" and limit it to a given interval
15:34:46 <dom> ... for the "unmute hint" use case, you don't need to know how long the person is speaking, and it doesn't need to be super frequent
15:35:12 <dom> Jianjun: this works for the 1st use case, but wouldn't help for the 2nd use case where you need to know when to stop and start processing
15:35:54 <dom> Tony: would a simple boolean suffice to give confidence on when to show the hint? e.g. clearly speaking vs maybe speech in the background
15:36:26 <dom> Jianjun: not sure - today looking for support for having such an API or not
15:36:54 <dom> TimP: we need to be clearer on whether this is voice detection or audio levels - they come with different requirements
15:37:07 <dom> ... the 1st one is specifically about voice, and the more specific the better
15:37:18 <dom> ... the 2nd one would want to work with singing, groups of people, etc
15:37:22 <dom> ... this feels like 2 APIs
15:38:09 <dom> Jianjun: I'm hearing the 2nd use case needs more thinking
15:38:32 <dom> Jan-Ivar: +1 on these being separate use cases that require separate situations
15:38:53 <dom> ... The 1st one is a single event that fires when the microphone is muted and there is voice activity
15:39:14 <dom> ... The 2nd one might be too noisy an an event on the main thread; it feels like metadata to attach to the audio, outside of the main thread
15:39:30 <dom> Jianjun: I'm happy to focus on the 1st use case atm
15:40:12 <dom> Youenn: +1 to Jan-Ivar; 1st use cases focused on muted track, 2nd on unmuted; the 2nd could be in audio stats, although it may be more usefully exposed in the audio worklet
15:40:42 <dom> ... in any case, separating them, doing the 1st one quickly and taking more time on the audio processing optimization
15:41:34 <dom> Harald: you could already skip processing audio frames with only zeros, although that itself introduces some overhead
15:42:11 <dom> Jianjun: I'm hearing we should focus on the 1st use case, and look into how to solve the 2nd use case separately
15:42:47 <dom> RESOLVED: proceed with a pull request for the 1st use case, and open a separate issue for optimizing audio processing
15:42:56 <dom> Subtopic: Issue #149 How to select camera presets that have better power efficiency at the expense of quality?
15:42:56 <dom> [slide 16]
15:43:30 <dom> RRSAgent, draft minutes
15:43:31 <RRSAgent> I have made the request to generate https://www.w3.org/2024/06/18-webrtc-minutes.html dom
15:45:20 <dom> [slide 17]
15:45:38 <dom> Present+ PeterThatcher
15:47:38 <dom> Dom: note that afaict there are no implementations of powerEfficientPixelFormat, so we could replace it altogether at this stage without backwards compat
15:48:14 <dom> Bernard: would powerEfficient always include the power efficient pixel format? if not, it may be useful to retain the granularity
15:48:48 <dom> Harald: this reminds me of content-hint - you don't want to specify exactly what to do, but pushing the UA in a certain direction
15:49:21 <dom> ... pixelFormat feel too constrainted, a more general one to replace it sounds like a good idea
15:49:31 <dom> ... e.g. a powerQualityTradeoff constraint
15:50:18 <dom> Bernard: the pixelFormat feels different than the general power efficiency
15:50:42 <dom> TimP: any reason the default should not be "powerEfficient"?
15:51:10 <dom> Youenn: this may break existing expectations e.g. for bar code scanners, high quality podcast recordings
15:51:42 <dom> ... it may be ideal to have that constraint default to true eventually, but that may be hard to do in a backward compatible manner, at least initially
15:52:08 <dom> ... I'm hearing support for removing powerEfficientPixelFormat and move toward a more general powerEfficient constraint
15:52:55 <dom> Jianlin: if two pages request a camera, one with powerEfficient and the other not, what would happen?
15:53:28 <dom> Youenn: e.g. the UA could re-size the video, with a bit of a quality loss
15:53:52 <dom> ... My thinking is "short usage, quality; longer, power efficiency"
15:54:05 <dom> Jan-Ivar: impact of power efficiency on barcode scanning?
15:54:16 <dom> Youenn: e.g. the autofocus might be more finicky in power efficient setting
15:55:27 <dom> Guido: we could start by adding a new constraint and reconsider removal of pixelFormat later - we don't have to do that at the same time
15:55:54 <dom> RESOLVED: Proceed with a pull request to add a powerEfficient constraint
15:56:20 <dom> Topic: -> https://github.com/w3c/webrtc-extensions/issues/209 ICE improvements: send and prevent ICE check over a candidate pair
15:56:20 <dom> [slide 20]
15:57:43 <dom> [slide 21]
15:59:44 <dom> Bernard: is consent freshness included in your concept of ICE check here?
15:59:51 <dom> ... if so, it may raise security issues
16:00:00 <dom> Sameer: I'll try to address that in the discussion of the API
16:00:52 <dom> ... the API I want to propose addresses connectivity checks, keepalive, and RTT determination, while conserving bandwidth & power
16:01:02 <dom> [slide 22]
16:03:59 <dom> Bernard: re consent freshness: is it considered a triggered check? it's good that responses can't be prevent, but can the app prevent consent freshness to go out?
16:04:16 <dom> Sameer: don't know off the top of my head, but +1 on making sure the app can't prevent it
16:05:06 <dom> Jan-Ivar: can you describe the use cases to expose the API to JS?
16:05:19 <dom> Sameer: this is more detailed in the issue description on github
16:05:50 <dom> ... STUN checks are sent for keepalive on the selected pair every 2s or so, but only every 15s or so on non-selected pairs
16:06:20 <dom> ... this API allows to monitor the quality of different candidate pairs and possibly switch e.g. if the RTT is lower
16:07:03 <dom> ... we've tried this out on mobile and seen very different RTT over the life of a session, across different network interfaces for instance
16:07:43 <dom> Jan-Ivar: there is no way for the UA to be doing this on its own?
16:08:13 <dom> Sameer: there is no obligation to the UA to be monitoring quality over ICE candidates
16:08:51 <dom> Jan-Ivar: could this be a IceTransport option instead though? instead of lots of new API surface
16:09:40 <dom> Sameer: picking an alternative may be hard to specify as a configuration (e.g. if you want to avoid relays due to latency)
16:10:07 <dom> Harald: the UA cannot know what options the application might have available that are not immediately visible
16:10:27 <dom> ... the purpose is the to allow the app to control the use of connections and figure the information it can get
16:10:40 <dom> ... the UA algorithms have to evolve slowly and work well in the general case
16:10:56 <dom> ... this API will allow experiemntation and considerations of things the UA cannot know
16:11:20 <dom> ... it's a movement towards ensuring the app can do what it wants to do while making the default behavior work well
16:12:01 <dom> Peter: re consent checks - I think it would be fine to disable consent checks, this would only mean that it would stop the ICE transport
16:12:14 <dom> ... (i.e. it wouldn't be unsafe, but a bit of a footgun)
16:12:34 <dom> ... in terms of the why, it would be very difficult for the UA to know what to do or to express it as a configuration
16:12:44 <dom> [slide 23]
16:16:08 <dom> [slide 24]
16:18:55 <dom> Sameer: the event-based API doesn't tell you when a check was sent if initiated by the ICE agent - the app may thus get a failure it tries to send a check too soon after the ICE agent
16:19:15 <dom> [slide 25]
16:20:17 <dom> TimP: there seems to be a big difference in scope, in terms of the context of what the app will see
16:20:31 <dom> ... the promises carries a lot of the context the event doesn't
16:21:02 <dom> ... e.g. whether the response comes from its own request or not - that feels like useful information to have - maybe this could be added to the event as well
16:21:52 <dom> Sameer: the app could determine whether they sent it - if they got a failure trying to request a check, they can know the one in flight comes from the agent
16:22:09 <dom> ... since there can only be one check in flight for a given pair
16:23:14 <dom> ... we could also expose the transaction id in the return value of the request (since it is exposed in the event)
16:23:40 <dom> Jan-Ivar: RTCIceCandidatePair is now an interface - the check method could be moved there
16:23:56 <dom> ... what is the transactionId array buffer
16:24:28 <dom> Sameer: it's the transaction id of the ICE check itself
16:24:54 <dom> Peter: it's 20 bytes in the STUN header that is useful for debugging, it's unique to each check
16:25:39 <dom> Jan-Ivar: so the proposal is a check method that the UA has done an ICE check on behalf of the app; could you cause network spam by running this in a loop?
16:26:19 <dom> Sameer: rate limiting is called out; since the main use case is to determine RTT / inactive pairs, 1s interval should be sufficient
16:27:28 <dom> Jan-Ivar: I would go with the promise approach - the event approach seems to be carrying a lot of state, which goes against general API advice
16:28:08 <dom> Peter: what happens if an ICE check goes lost? is there a timeout to determine it is no longer in flight?
16:28:31 <dom> Sameer: when a check is sent out, it's marked as in progress; if it timeoutes, it's marked as failed
16:28:35 <dom> s/tes/ts/
16:29:20 <dom> Peter: it doesn't allow the app to decide what the timeout should be; and we would want more than one check in flight in general as during establishments
16:29:41 <dom> ... as long as the browser is exposing rate limits, this should be OK
16:30:25 <dom> Jan-Ivar: re event API shape, maybe the values could be exposed on the icecandidatepair instead of the event
16:30:54 <dom> Sameer: this would be similar to what we can get with getStats - I think it's more useful to expose immediate feedback with the response
16:32:02 <dom> Sameer: overall, I'm hearing support for the promises approach, and investigating multiple in-flight checks at the same time
16:32:11 <dom> Topic: -> https://github.com/w3c/webrtc-pc/issues/2977 WebRTC: PC.local_description and friends - snapshot views or dynamic views?
16:32:11 <dom> [slide 29]
16:34:30 <dom> RESOLVED: no objection to proposal to align spec with implementations
16:34:47 <dom> Topic: -> https://github.com/w3c/mediacapture-main/issues/966 Should devicechange fire when the device info changes?
16:34:47 <dom> [slide 30]
16:35:19 <dom> [slide 31]
16:36:56 <dom> RRSAgent, draft minutes
16:36:58 <RRSAgent> I have made the request to generate https://www.w3.org/2024/06/18-webrtc-minutes.html dom
16:39:21 <dom> TimP: how about most recently inserted device?
16:39:34 <dom> Jan-Ivar: for it to be a strong signal, it has to be recent
16:39:46 <dom> ... also there can be several devices (e.g. a camera that sports a mic)
16:40:18 <dom> ... but we can bikeshed the name on the issue
16:40:42 <dom> ... or are you thinking of a different logic?
16:40:56 <dom> TimP: mostly suggesting a way to simplify the explanation of the behavior
16:41:27 <dom> Jan-Ivar: inserted may have connotations that are no longer accurate for all devices, but has a nice intent aspect to it
16:41:45 <dom> harald: it's a new feature - how can you detect it?
16:42:02 <dom> Jan-Ivar: you get an empty array if no devices were inserted (vs no property if not implemented)
16:42:23 <dom> RESOLVED: Continue discussion on PR towards merging
16:42:26 <dom> RRSAgent, draft minutes
16:42:27 <RRSAgent> I have made the request to generate https://www.w3.org/2024/06/18-webrtc-minutes.html dom
18:36:18 <Zakim> Zakim has left #webrtc