16:00:27 <RRSAgent> RRSAgent has joined #webrtc
16:00:32 <RRSAgent> logging to https://www.w3.org/2023/12/12-webrtc-irc
16:00:32 <dom> Meeting: WebRTC December 12 2023 meeting
16:00:32 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/December_12_2023
16:00:32 <dom> Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf
16:00:32 <dom> Chairs: HTA, Jan-Ivar, Bernard
16:00:45 <dom> Present+ Dom, Youenn, Tove, Elad, Harald, Bernard, TimPanton, Guido
16:03:06 <dom> Present+ Fippo
16:03:10 <dom> Recording is starting
16:03:10 <dom> Recording: https://youtu.be/n0PeqoRslXs
16:03:17 <dom> Present+ PeterThatcher
16:05:38 <dom> Topic: -> https://github.com/w3c/mediacapture-screen-share Screen Capture
16:05:49 <dom> Subtopic: Issue #276: Handling of contradictory hints
16:05:49 <dom> [slide 11]
16:06:15 <dom> [slide 12]
16:06:41 <dom> [slide 13]
16:06:58 <dom> Present+ Varun
16:07:03 <dom> [slide 14]
16:07:46 <dom> Present+ Jan-Ivar
16:07:55 <dom> [+1s from fippo, bernard, harald]
16:08:20 <dom> Jan-Ivar: this shows maybe we went a bit fast with these hints; the UA is free not to react to them
16:08:36 <dom> ... I would just ignore them
16:08:49 <dom> Bernard: I've noticed this pattern with hints in other places
16:09:02 <dom> ... it's problematic - this feels like bad config
16:09:08 <dom> ... maybe they should not be hints
16:09:34 <dom> elad: certain constraints can be self-contradictory leading to overconstrained error
16:10:31 <dom> Timp: what's the goal here? notify the developer they set an incompatible set of options? protecting the user from something bad?
16:10:50 <dom> Elad: what error should be thrown interoperably?
16:11:00 <dom> Timp: but what would happen with the error thrown?
16:11:20 <dom> Elad: the developer should fix their code
16:11:41 <dom> TimP: you want it to fail then?
16:12:05 <dom> ... this is probably the right thing to do
16:12:21 <dom> Elad: having fail it the same across browsers would be good
16:12:59 <dom> Harald: +1 for specifying something; I don't it matters so much what is specified - ignoring may be OK in some cases
16:13:23 <dom> Elad: +1 to focus on interop first and foremost
16:13:51 <dom> bernard: some tricky aspects: given the goal is to guide developers, how would they be guided toward their mistake?
16:14:26 <dom> ... if you ignore or reject, it needs to be clear to the developer what was ignored/rejected, and what remains in the end
16:15:11 <dom> Elad: the error could point to the list of things allowed/disallowed
16:15:28 <dom> ... ignoring the hints makes it probably trickier to avoid unexpected results
16:15:37 <dom> Bernard: can you retrieve what was eventually applied?
16:15:44 <dom> Elad: if you reject, nothing is applied
16:15:49 <dom> Bernard: but what if you ignore?
16:15:56 <dom> Elad: hence why I think rejecting is the right thing
16:16:15 <dom> Youenn: normally we try to minimize API to avoid contradictory choices - we should avoid that moving forward
16:16:31 <dom> ... here, rejecting to get interop is OK
16:17:46 <dom> Jan-Ivar: we have to take into account display switching (as we will discuss)
16:17:57 <dom> ... looking at a PR will help
16:18:27 <dom> Elad: the PR for display switching has some discussion on how it is impacted by constraints
16:19:05 <dom> RESOLVED: Consensus to specify an interoperable behavior and iterate initially on a pull request to be proposed by Elad
16:19:15 <dom> Subtopic: Issue #281: Distinguish cancellations from absent OS permissions
16:19:15 <dom> [slide 15]
16:19:22 <dom> RRSAgent, draft minutes
16:19:23 <RRSAgent> I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
16:19:35 <dom> [slide 16]
16:20:13 <dom> Elad: Jan-Ivar mentioned that permission.query might serve this purpose; I wonder if it introduces a fingerprinting concern
16:20:56 <dom> Jan-Ivar: you're right that it would, but the mid-term solution proposed in the issue would match mitigation for permission.query, so I think we could make that work
16:21:35 <dom> Elad: would that work with Safari in embargo mode?
16:21:48 <dom> ... when the user cancels 3 times a call
16:22:12 <dom> Youenn: this is under the control of the UA; it could report "blocked" under these circumstances
16:22:46 <dom> Elad: but would there be a way to distinguish OS-blocked vs user-blocked?
16:24:12 <dom> Harald: if we want to have distinguishable situations, there needs to be matching API surface
16:24:29 <dom> Youenn: I'm not sure embargo-mode or not is relevant
16:24:40 <dom> ... what Elad is asking for is to tell the user that "something happened"
16:24:52 <dom> ... if we have more than OS-rejected vs user-reject, an enum is needed
16:24:59 <dom> ... otherwise, the permission API may be enough
16:25:14 <dom> ... it would still need to be specified given that permission.query is still a bit fuzzy
16:25:23 <dom> ... do you foresee more values in the enum?
16:25:29 <dom> Elad: another problematic scenario
16:26:13 <dom> ... an app being relaunched may not be able to determine if it's blocked because of the user vs OS without calling getDisplayMedia
16:26:19 <dom> Youenn: that's true of your proposal as well
16:26:31 <dom> ... so for the enum, do you see more values?
16:26:42 <dom> Elad: a boolean may be OK but harder to extend
16:27:33 <dom> ... the new API I propose would work better e.g. if a browser in the future decides to embargo permission after a single call
16:28:16 <dom> Jan-Ivar: I would object going further to permission.query - we shouldn't reveal an OS level decision, the user agent is the party
16:28:38 <dom> Elad: how about boolean "user-rejected" yes or no?
16:28:46 <dom> Jan-Ivar: that's equivalent to permission.query
16:29:25 <dom> Elad: the user won't know what they can do to solve the situation
16:29:31 <dom> Jan-Ivar: that's up to the UA to guide them
16:29:51 <dom> Elad: I think we should empower the app to help the user instead of always relying on the UA
16:30:04 <dom> Youenn: this isn't specific to getDisplayMedia - this would probably apply to mic and cameras
16:30:15 <dom> ... how are we dealing with this?
16:30:37 <dom> ... if it's a problem worth solving, it should be solved across sources
16:30:58 <dom> Present+ TonyHerre
16:31:16 <dom> Elad: this solution could be extended to getUserMedia
16:31:34 <dom> Youenn: I think we should explore permission.query - if it works for gDM, it would work for gUM
16:32:00 <dom> Elad: I'll explore this, although I think the embargo issue will be problem
16:32:28 <dom> harald: the user operates with the UA & OS separately, and the UA and OS aren't friends - there needs to be a way to guide the user toward the OS
16:32:50 <dom> ... a query based system can only tell you about the situation as it is now; the error feels like a superior situation
16:33:06 <dom> Elad: permission.query is async - so indeed the answer may no longer the right one
16:33:20 <dom> [skipping issue 219]
16:33:25 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/issues/39 Solve user agent camera/microphone double-mute (mediacapture-extensions)
16:33:26 <dom> [slide 25]
16:34:15 <dom> [slide 26]
16:34:34 <dom> [slide 27]
16:35:01 <dom> [slide 28]
16:35:27 <dom> [slide 29]
16:36:13 <dom> [slide 30]
16:36:57 <dom> [slide 31]
16:37:50 <dom> [slide 32]
16:38:54 <dom> [jan-ivar points to https://github.com/w3c/mediacapture-main/issues/982 ]
16:39:34 <dom> [slide 33]
16:41:36 <dom> [slide 34]
16:42:36 <dom> [jan-ivar points to https://github.com/w3c/mediasession/issues/279]
16:43:16 <dom> [hta, aboba +1 MuteReason on chat]
16:43:35 <dom> jan-ivar: the 1st thing we should is to make it clear where the discussion is happening on github
16:43:52 <dom> ... I didn't feel like the slides were representative of the discussion on github
16:44:05 <dom> ... I see the same issue with MuteReason as what I described earlier
16:44:37 <dom> ... I don't think we should expose an OS setting
16:45:21 <dom> ... there are cases where the UA might be muting - which is why I opened an issue to explore that question in https://github.com/w3c/mediacapture-main/issues/982 which shows different interpretation by UAs of muted state
16:46:08 <dom> Elad: I want to impress on everybody that this is an important issue for users and developers, and brought critique on alternative solutions that had been suggested
16:46:26 <dom> Youenn: there is an existing solution with the MediaSession API that is shipping in Chrome
16:46:47 <dom> ... we need to discuss with them whether this will solve this issue or not, and if not, we need to understand the differences
16:47:02 <dom> ... it may be that we could fix or extend the mediasession API
16:47:11 <dom> ... we need to understand the relationship between the two no matter what
16:47:32 <dom> ... I also agree with Jan-Ivar we need to clarify the meaning of mute vs end - this is hurting developers
16:47:39 <dom> ... this feels as important as this issue
16:48:21 <dom> ... there are interoperability issues in end vs mute - they're all different across browsers
16:48:52 <dom> ... I would like to start a discussion with the Media WG to understand the interactions between track.muted and MediaSession
16:49:08 <dom> Elad: we've looked into this and it didn't look like a compelling solution
16:49:54 <dom> Bernard: when the app controls mute, it can inform the user you're speaking while muted
16:49:54 <dom> ... but it can't do that when the OS is in control; what would you do with MuteReason?
16:50:38 <dom> Youenn: the track would be muted, but you would still be able through a separate event to notify the app that the user is speaking
16:51:06 <dom> Elad: but if the user is not trying to speak, the user would still not know they're OS-muted in the app UI
16:51:13 <dom> Bernard: how would you surface this?
16:51:38 <dom> Elad: the app could reflect that via multiple UI states, or reflect the latest detected change, or give a bit more context in the UI
16:52:26 <dom> HTA: I tried in Chrome: setMicrophoneActive: true doesn't unmute, setMicrophoneActive: false doesn't mute
16:52:50 <dom> ... re end vs mute - if they're diverging behavior, we should fix that - in implementations or specs, as needed
16:52:55 <dom> ... but this is orthogonal
16:53:22 <dom> Jan-Ivar: the app receives enabled/muted already - MuteReason doesn't address that
16:53:44 <dom> ... the MediaSession API provides an API for applications that have mute toggles to expand those to keyboard, hardware, lock screen, etc
16:54:22 <dom> ... there is no muting in MediaSession at the moment
16:54:33 <dom> ... the UA is already allowed to mute at any point
16:54:46 <dom> Youenn: we do need to talk about the intents of the MediaSession API
16:55:06 <dom> Elad: we've shown a PR of what MuteReason could do; we haven't heard why we shouldn't expose an OS setting
16:55:21 <dom> ... our own privacy review has qualified this as benign
16:55:35 <dom> ... I suspect solving this with MediaSession will be complex, but happy to be proved wrong
16:55:45 <dom> ... Hope we can make progress on this before the next meeting
16:55:49 <dom> Topic: Dynamic Switching in Captured Surfaces (Tove)
16:55:49 <dom> [slide 37]
16:56:00 <dom> RRSAgent, draft minutes
16:56:01 <RRSAgent> I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
16:56:06 <dom> RRSAgent, make log public
16:56:29 <dom> [slide 38]
16:57:50 <dom> [slide 39]
16:59:01 <dom> [slide 40]
16:59:29 <dom> [slide 41]
17:00:53 <dom> s|Topic:|Topic: -> https://github.com/w3c/mediacapture-screen-share/issues/255
17:01:00 <dom> RRSAgent, draft minutes
17:01:01 <RRSAgent> I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
17:01:19 <dom> s/(Tove)//
17:01:24 <dom> [slide 42]
17:02:40 <dom> [slide 43]
17:05:21 <dom> jan-ivar: good summary of the discussion, thanks! looking at slide 40, would sourceswitch fire if there is no opt-in?
17:05:36 <dom> Trove: I think we could
17:05:52 <dom> jan-ivar: would the event come with the same stream in that situation?
17:06:10 <dom> Trove: it will be a new stream
17:06:35 <dom> Jan-Ivar: I'm in favor of the late decision model: you get the event regardless you opted in to assisted switching
17:06:57 <dom> .... having the injection model as the default, and preventing it with preventDefault() is aligned with well-known Web platform patterns
17:07:39 <dom> Elad: looking at the code slide 42, I find it hard to understand what preventDefault() does
17:08:04 <dom> ... and if there is not preventDefault(), it's even less clear that something default is happening
17:08:09 <dom> ... this feels foot-gunny
17:08:38 <dom> ... what purpose does this serve? What app needs to make a late decision? it feels like app would always want one or the other model
17:09:06 <dom> Youenn: I wasn't sure whether surface switching with injection model was good or not
17:09:25 <dom> ... I think it's good to support surface type change - we have the configuration event for that
17:09:52 <dom> ... allowing apps to optimize it is good; late-switching is a nice-to-have, but not required if it's particularly difficult to implement
17:09:59 <dom> ... but having web developers flexibility is nice
17:10:18 <dom> Elad: you've often mentioned that some complexity is to the detriment of the developer, which seems to be the case here
17:10:47 <dom> Jan-Ivar: the complexity comes from the fact that this is changing an existing shipped behavior - we can't avoid it
17:11:27 <dom> ... no matter what solution we choose, the default behavior won't be obvious
17:11:36 <dom> ... I think it's better to fall back on the injection model
17:12:03 <dom> Trove: it's hard to know *when* you need to switch tracks
17:12:15 <dom> ... it affects capabilities, it may affect which methods can be called
17:13:05 <dom> jan-ivar: the question is when the decision happens - the app chooses
17:14:22 <dom> ... a late decision can allow to minimize the glitch by detecting what kind of replacement is happening
17:14:27 <dom> Elad: shouldn't the UA fix that?
17:15:02 <dom> jan-ivar: the UA can't fix this - e.g. with replaceTrack because of downstream consequences e.g. in MediaRecorder
17:15:32 <dom> Elad: could we demonstrate a path forward for a later addition of late decision?
17:15:39 <dom> Jan-Ivar: -1
17:16:14 <dom> Topic: -> https://github.com/w3c/webrtc-encoded-transform/issues/211 RtpSender Encoded Source
17:16:19 <dom> [slide 46]
17:16:54 <dom> Present+ PatrickRockhill
17:17:17 <dom> Slide [47]
17:17:56 <dom> Slide [48]
17:18:51 <dom> Slide [49]
17:20:48 <dom> Slide [50]
17:21:59 <dom> Slide [51]
17:23:00 <dom> Youenn: in general, we want to design the API to allow plugging an encoder
17:23:26 <dom> ... then we add specific constraints for cases where sources and encoders are combined
17:23:45 <dom> ... that would be my general advice
17:24:07 <dom> ... I would be in favor to have immutable objects in general
17:24:16 <dom> Guido: so constructor over setMetadata?
17:24:27 <dom> Youenn: yes - that has been the approach with WebCodecs
17:24:34 <dom> Guido: I can agree with that
17:24:51 <dom> ... For the forwarding use case, you already have frames that you are forwarding
17:25:28 <dom> Harald: we have an agreed upon use case, the fan out, but we don't have one for encoders
17:26:09 <dom> ... I kind of agree treating frames as immutable is better if we can solve the @@@ issue
17:26:37 <dom> ... but for this use case, handling RTP-relevant data is what matters, so I don't see the need for a new object
17:26:53 <dom> Jan-Ivar: we've heard many proposals for ways to expose the media pipeline to JS
17:27:06 <dom> ... via transforms
17:27:17 <dom> Guido: this is not for encoding, but forwarding already encoded frames
17:27:39 <dom> Jan-Ivar: but does this not also allow JS to get frames from anywhere?
17:27:52 <dom> Guido: from another PC?
17:28:14 <dom> Jan-Ivar: Youenn's proposal allowed to get frames from WebCodecs
17:28:39 <dom> Guido: yes, but in that case they don't have the RTP metadata that would allow forwarding
17:28:52 <dom> Jan-Ivar: but I'm not sure there is consensus on Youenn's proposal
17:29:27 <dom> ... re [slide 48], where would be RTCRtpSenderEncodedSource exposed?
17:29:45 <dom> Guido: I think Youenn meant to expose on Worker only, with the handle being transferable
17:29:55 <dom> Jan-Ivar: happy to hear that
17:30:44 <dom> [TimP, Harald gives +1 to the approach]
17:31:07 <dom> Bernard: the RTCRTpSenderEncodedSource - do we really need two enqueue methods?
17:31:16 <dom> Guido: we only need one for the forwarding use case
17:31:41 <dom> Bernard: can we convert between audiochunk and and rtpchunk?
17:31:52 <dom> ... this could simplify the API
17:32:04 <dom> Guido: there isn't such a way at the moment, but we could look into one
17:32:50 <dom> Harald: a constructor with a chunk as input could do this, and avoids touching rtpsenderencodedsource
17:33:24 <dom> Jan-Ivar: I think it may be a good direction, but I would love to see JS code that shows where the encoded frames are coming from
17:33:30 <dom> Guido: [shows slide 47]
17:33:44 <dom> Jan-Ivar: is there a receiver equivalent to this?
17:33:50 <dom> Guido: that would be the logical follow up to this
17:34:25 <dom> ... e.g. to turn multiple input into a more reliable input for playback
17:35:00 <dom> RESOLVED: seems like a promising direction for which to see a more complete proposal
17:35:05 <dom> Topic: -> https://github.com/w3c/webrtc-encoded-transform/pull/215 Keyframe API
17:35:07 <dom> [slide 54]
17:36:04 <dom> Harald: #215 is merged - we now have a spec for a keyframe event
17:36:09 <dom> [slide 55]
17:37:19 <dom> [slide 56]
17:38:40 <dom> [slide 57]
17:40:11 <dom> Jan-Ivar: +1 - hoping we can present API proposals next time around
17:40:38 <dom> RRSAgent, draft minutes
17:40:40 <RRSAgent> I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom