IRC log of webrtc on 2023-12-12

Timestamps are in UTC.

16:00:27 [RRSAgent]
RRSAgent has joined #webrtc
16:00:32 [RRSAgent]
logging to https://www.w3.org/2023/12/12-webrtc-irc
16:00:32 [dom]
Meeting: WebRTC December 12 2023 meeting
16:00:32 [dom]
Agenda: https://www.w3.org/2011/04/webrtc/wiki/December_12_2023
16:00:32 [dom]
Slideset: https://lists.w3.org/Archives/Public/www-archive/2023Dec/att-0002/WEBRTCWG-2023-12-12__1_.pdf
16:00:32 [dom]
Chairs: HTA, Jan-Ivar, Bernard
16:00:45 [dom]
Present+ Dom, Youenn, Tove, Elad, Harald, Bernard, TimPanton, Guido
16:03:06 [dom]
Present+ Fippo
16:03:10 [dom]
Recording is starting
16:03:17 [dom]
Present+ PeterThatcher
16:05:38 [dom]
Topic: -> https://github.com/w3c/mediacapture-screen-share Screen Capture
16:05:49 [dom]
Subtopic: Issue #276: Handling of contradictory hints
16:05:49 [dom]
[slide 11]
16:06:15 [dom]
[slide 12]
16:06:41 [dom]
[slide 13]
16:06:58 [dom]
Present+ Varun
16:07:03 [dom]
[slide 14]
16:07:46 [dom]
Present+ Jan-Ivar
16:07:55 [dom]
[+1s from fippo, bernard, harald]
16:08:20 [dom]
Jan-Ivar: this shows maybe we went a bit fast with these hints; the UA is free not to react to them
16:08:36 [dom]
... I would just ignore them
16:08:49 [dom]
Bernard: I've noticed this pattern with hints in other places
16:09:02 [dom]
... it's problematic - this feels like bad config
16:09:08 [dom]
... maybe they should not be hints
16:09:34 [dom]
elad: certain constraints can be self-contradictory leading to overconstrained error
16:10:31 [dom]
Timp: what's the goal here? notify the developer they set an incompatible set of options? protecting the user from something bad?
16:10:50 [dom]
Elad: what error should be thrown interoperably?
16:11:00 [dom]
Timp: but what would happen with the error thrown?
16:11:20 [dom]
Elad: the developer should fix their code
16:11:41 [dom]
TimP: you want it to fail then?
16:12:05 [dom]
... this is probably the right thing to do
16:12:21 [dom]
Elad: having fail it the same across browsers would be good
16:12:59 [dom]
Harald: +1 for specifying something; I don't it matters so much what is specified - ignoring may be OK in some cases
16:13:23 [dom]
Elad: +1 to focus on interop first and foremost
16:13:51 [dom]
bernard: some tricky aspects: given the goal is to guide developers, how would they be guided toward their mistake?
16:14:26 [dom]
... if you ignore or reject, it needs to be clear to the developer what was ignored/rejected, and what remains in the end
16:15:11 [dom]
Elad: the error could point to the list of things allowed/disallowed
16:15:28 [dom]
... ignoring the hints makes it probably trickier to avoid unexpected results
16:15:37 [dom]
Bernard: can you retrieve what was eventually applied?
16:15:44 [dom]
Elad: if you reject, nothing is applied
16:15:49 [dom]
Bernard: but what if you ignore?
16:15:56 [dom]
Elad: hence why I think rejecting is the right thing
16:16:15 [dom]
Youenn: normally we try to minimize API to avoid contradictory choices - we should avoid that moving forward
16:16:31 [dom]
... here, rejecting to get interop is OK
16:17:46 [dom]
Jan-Ivar: we have to take into account display switching (as we will discuss)
16:17:57 [dom]
... looking at a PR will help
16:18:27 [dom]
Elad: the PR for display switching has some discussion on how it is impacted by constraints
16:19:05 [dom]
RESOLVED: Consensus to specify an interoperable behavior and iterate initially on a pull request to be proposed by Elad
16:19:15 [dom]
Subtopic: Issue #281: Distinguish cancellations from absent OS permissions
16:19:15 [dom]
[slide 15]
16:19:22 [dom]
RRSAgent, draft minutes
16:19:23 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
16:19:35 [dom]
[slide 16]
16:20:13 [dom]
Elad: Jan-Ivar mentioned that permission.query might serve this purpose; I wonder if it introduces a fingerprinting concern
16:20:56 [dom]
Jan-Ivar: you're right that it would, but the mid-term solution proposed in the issue would match mitigation for permission.query, so I think we could make that work
16:21:35 [dom]
Elad: would that work with Safari in embargo mode?
16:21:48 [dom]
... when the user cancels 3 times a call
16:22:12 [dom]
Youenn: this is under the control of the UA; it could report "blocked" under these circumstances
16:22:46 [dom]
Elad: but would there be a way to distinguish OS-blocked vs user-blocked?
16:24:12 [dom]
Harald: if we want to have distinguishable situations, there needs to be matching API surface
16:24:29 [dom]
Youenn: I'm not sure embargo-mode or not is relevant
16:24:40 [dom]
... what Elad is asking for is to tell the user that "something happened"
16:24:52 [dom]
... if we have more than OS-rejected vs user-reject, an enum is needed
16:24:59 [dom]
... otherwise, the permission API may be enough
16:25:14 [dom]
... it would still need to be specified given that permission.query is still a bit fuzzy
16:25:23 [dom]
... do you foresee more values in the enum?
16:25:29 [dom]
Elad: another problematic scenario
16:26:13 [dom]
... an app being relaunched may not be able to determine if it's blocked because of the user vs OS without calling getDisplayMedia
16:26:19 [dom]
Youenn: that's true of your proposal as well
16:26:31 [dom]
... so for the enum, do you see more values?
16:26:42 [dom]
Elad: a boolean may be OK but harder to extend
16:27:33 [dom]
... the new API I propose would work better e.g. if a browser in the future decides to embargo permission after a single call
16:28:16 [dom]
Jan-Ivar: I would object going further to permission.query - we shouldn't reveal an OS level decision, the user agent is the party
16:28:38 [dom]
Elad: how about boolean "user-rejected" yes or no?
16:28:46 [dom]
Jan-Ivar: that's equivalent to permission.query
16:29:25 [dom]
Elad: the user won't know what they can do to solve the situation
16:29:31 [dom]
Jan-Ivar: that's up to the UA to guide them
16:29:51 [dom]
Elad: I think we should empower the app to help the user instead of always relying on the UA
16:30:04 [dom]
Youenn: this isn't specific to getDisplayMedia - this would probably apply to mic and cameras
16:30:15 [dom]
... how are we dealing with this?
16:30:37 [dom]
... if it's a problem worth solving, it should be solved across sources
16:30:58 [dom]
Present+ TonyHerre
16:31:16 [dom]
Elad: this solution could be extended to getUserMedia
16:31:34 [dom]
Youenn: I think we should explore permission.query - if it works for gDM, it would work for gUM
16:32:00 [dom]
Elad: I'll explore this, although I think the embargo issue will be problem
16:32:28 [dom]
harald: the user operates with the UA & OS separately, and the UA and OS aren't friends - there needs to be a way to guide the user toward the OS
16:32:50 [dom]
... a query based system can only tell you about the situation as it is now; the error feels like a superior situation
16:33:06 [dom]
Elad: permission.query is async - so indeed the answer may no longer the right one
16:33:20 [dom]
[skipping issue 219]
16:33:25 [dom]
Topic: -> https://github.com/w3c/mediacapture-extensions/issues/39 Solve user agent camera/microphone double-mute (mediacapture-extensions)
16:33:26 [dom]
[slide 25]
16:34:15 [dom]
[slide 26]
16:34:34 [dom]
[slide 27]
16:35:01 [dom]
[slide 28]
16:35:27 [dom]
[slide 29]
16:36:13 [dom]
[slide 30]
16:36:57 [dom]
[slide 31]
16:37:50 [dom]
[slide 32]
16:38:54 [dom]
[jan-ivar points to https://github.com/w3c/mediacapture-main/issues/982 ]
16:39:34 [dom]
[slide 33]
16:41:36 [dom]
[slide 34]
16:42:36 [dom]
[jan-ivar points to https://github.com/w3c/mediasession/issues/279]
16:43:16 [dom]
[hta, aboba +1 MuteReason on chat]
16:43:35 [dom]
jan-ivar: the 1st thing we should is to make it clear where the discussion is happening on github
16:43:52 [dom]
... I didn't feel like the slides were representative of the discussion on github
16:44:05 [dom]
... I see the same issue with MuteReason as what I described earlier
16:44:37 [dom]
... I don't think we should expose an OS setting
16:45:21 [dom]
... there are cases where the UA might be muting - which is why I opened an issue to explore that question in https://github.com/w3c/mediacapture-main/issues/982 which shows different interpretation by UAs of muted state
16:46:08 [dom]
Elad: I want to impress on everybody that this is an important issue for users and developers, and brought critique on alternative solutions that had been suggested
16:46:26 [dom]
Youenn: there is an existing solution with the MediaSession API that is shipping in Chrome
16:46:47 [dom]
... we need to discuss with them whether this will solve this issue or not, and if not, we need to understand the differences
16:47:02 [dom]
... it may be that we could fix or extend the mediasession API
16:47:11 [dom]
... we need to understand the relationship between the two no matter what
16:47:32 [dom]
... I also agree with Jan-Ivar we need to clarify the meaning of mute vs end - this is hurting developers
16:47:39 [dom]
... this feels as important as this issue
16:48:21 [dom]
... there are interoperability issues in end vs mute - they're all different across browsers
16:48:52 [dom]
... I would like to start a discussion with the Media WG to understand the interactions between track.muted and MediaSession
16:49:08 [dom]
Elad: we've looked into this and it didn't look like a compelling solution
16:49:54 [dom]
Bernard: when the app controls mute, it can inform the user you're speaking while muted
16:49:54 [dom]
... but it can't do that when the OS is in control; what would you do with MuteReason?
16:50:38 [dom]
Youenn: the track would be muted, but you would still be able through a separate event to notify the app that the user is speaking
16:51:06 [dom]
Elad: but if the user is not trying to speak, the user would still not know they're OS-muted in the app UI
16:51:13 [dom]
Bernard: how would you surface this?
16:51:38 [dom]
Elad: the app could reflect that via multiple UI states, or reflect the latest detected change, or give a bit more context in the UI
16:52:26 [dom]
HTA: I tried in Chrome: setMicrophoneActive: true doesn't unmute, setMicrophoneActive: false doesn't mute
16:52:50 [dom]
... re end vs mute - if they're diverging behavior, we should fix that - in implementations or specs, as needed
16:52:55 [dom]
... but this is orthogonal
16:53:22 [dom]
Jan-Ivar: the app receives enabled/muted already - MuteReason doesn't address that
16:53:44 [dom]
... the MediaSession API provides an API for applications that have mute toggles to expand those to keyboard, hardware, lock screen, etc
16:54:22 [dom]
... there is no muting in MediaSession at the moment
16:54:33 [dom]
... the UA is already allowed to mute at any point
16:54:46 [dom]
Youenn: we do need to talk about the intents of the MediaSession API
16:55:06 [dom]
Elad: we've shown a PR of what MuteReason could do; we haven't heard why we shouldn't expose an OS setting
16:55:21 [dom]
... our own privacy review has qualified this as benign
16:55:35 [dom]
... I suspect solving this with MediaSession will be complex, but happy to be proved wrong
16:55:45 [dom]
... Hope we can make progress on this before the next meeting
16:55:49 [dom]
Topic: Dynamic Switching in Captured Surfaces (Tove)
16:55:49 [dom]
[slide 37]
16:56:00 [dom]
RRSAgent, draft minutes
16:56:01 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
16:56:06 [dom]
RRSAgent, make log public
16:56:29 [dom]
[slide 38]
16:57:50 [dom]
[slide 39]
16:59:01 [dom]
[slide 40]
16:59:29 [dom]
[slide 41]
17:00:53 [dom]
s|Topic:|Topic: -> https://github.com/w3c/mediacapture-screen-share/issues/255
17:01:00 [dom]
RRSAgent, draft minutes
17:01:01 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom
17:01:19 [dom]
s/(Tove)//
17:01:24 [dom]
[slide 42]
17:02:40 [dom]
[slide 43]
17:05:21 [dom]
jan-ivar: good summary of the discussion, thanks! looking at slide 40, would sourceswitch fire if there is no opt-in?
17:05:36 [dom]
Trove: I think we could
17:05:52 [dom]
jan-ivar: would the event come with the same stream in that situation?
17:06:10 [dom]
Trove: it will be a new stream
17:06:35 [dom]
Jan-Ivar: I'm in favor of the late decision model: you get the event regardless you opted in to assisted switching
17:06:57 [dom]
.... having the injection model as the default, and preventing it with preventDefault() is aligned with well-known Web platform patterns
17:07:39 [dom]
Elad: looking at the code slide 42, I find it hard to understand what preventDefault() does
17:08:04 [dom]
... and if there is not preventDefault(), it's even less clear that something default is happening
17:08:09 [dom]
... this feels foot-gunny
17:08:38 [dom]
... what purpose does this serve? What app needs to make a late decision? it feels like app would always want one or the other model
17:09:06 [dom]
Youenn: I wasn't sure whether surface switching with injection model was good or not
17:09:25 [dom]
... I think it's good to support surface type change - we have the configuration event for that
17:09:52 [dom]
... allowing apps to optimize it is good; late-switching is a nice-to-have, but not required if it's particularly difficult to implement
17:09:59 [dom]
... but having web developers flexibility is nice
17:10:18 [dom]
Elad: you've often mentioned that some complexity is to the detriment of the developer, which seems to be the case here
17:10:47 [dom]
Jan-Ivar: the complexity comes from the fact that this is changing an existing shipped behavior - we can't avoid it
17:11:27 [dom]
... no matter what solution we choose, the default behavior won't be obvious
17:11:36 [dom]
... I think it's better to fall back on the injection model
17:12:03 [dom]
Trove: it's hard to know *when* you need to switch tracks
17:12:15 [dom]
... it affects capabilities, it may affect which methods can be called
17:13:05 [dom]
jan-ivar: the question is when the decision happens - the app chooses
17:14:22 [dom]
... a late decision can allow to minimize the glitch by detecting what kind of replacement is happening
17:14:27 [dom]
Elad: shouldn't the UA fix that?
17:15:02 [dom]
jan-ivar: the UA can't fix this - e.g. with replaceTrack because of downstream consequences e.g. in MediaRecorder
17:15:32 [dom]
Elad: could we demonstrate a path forward for a later addition of late decision?
17:15:39 [dom]
Jan-Ivar: -1
17:16:14 [dom]
Topic: -> https://github.com/w3c/webrtc-encoded-transform/issues/211 RtpSender Encoded Source
17:16:19 [dom]
[slide 46]
17:16:54 [dom]
Present+ PatrickRockhill
17:17:17 [dom]
Slide [47]
17:17:56 [dom]
Slide [48]
17:18:51 [dom]
Slide [49]
17:20:48 [dom]
Slide [50]
17:21:59 [dom]
Slide [51]
17:23:00 [dom]
Youenn: in general, we want to design the API to allow plugging an encoder
17:23:26 [dom]
... then we add specific constraints for cases where sources and encoders are combined
17:23:45 [dom]
... that would be my general advice
17:24:07 [dom]
... I would be in favor to have immutable objects in general
17:24:16 [dom]
Guido: so constructor over setMetadata?
17:24:27 [dom]
Youenn: yes - that has been the approach with WebCodecs
17:24:34 [dom]
Guido: I can agree with that
17:24:51 [dom]
... For the forwarding use case, you already have frames that you are forwarding
17:25:28 [dom]
Harald: we have an agreed upon use case, the fan out, but we don't have one for encoders
17:26:09 [dom]
... I kind of agree treating frames as immutable is better if we can solve the @@@ issue
17:26:37 [dom]
... but for this use case, handling RTP-relevant data is what matters, so I don't see the need for a new object
17:26:53 [dom]
Jan-Ivar: we've heard many proposals for ways to expose the media pipeline to JS
17:27:06 [dom]
... via transforms
17:27:17 [dom]
Guido: this is not for encoding, but forwarding already encoded frames
17:27:39 [dom]
Jan-Ivar: but does this not also allow JS to get frames from anywhere?
17:27:52 [dom]
Guido: from another PC?
17:28:14 [dom]
Jan-Ivar: Youenn's proposal allowed to get frames from WebCodecs
17:28:39 [dom]
Guido: yes, but in that case they don't have the RTP metadata that would allow forwarding
17:28:52 [dom]
Jan-Ivar: but I'm not sure there is consensus on Youenn's proposal
17:29:27 [dom]
... re [slide 48], where would be RTCRtpSenderEncodedSource exposed?
17:29:45 [dom]
Guido: I think Youenn meant to expose on Worker only, with the handle being transferable
17:29:55 [dom]
Jan-Ivar: happy to hear that
17:30:44 [dom]
[TimP, Harald gives +1 to the approach]
17:31:07 [dom]
Bernard: the RTCRTpSenderEncodedSource - do we really need two enqueue methods?
17:31:16 [dom]
Guido: we only need one for the forwarding use case
17:31:41 [dom]
Bernard: can we convert between audiochunk and and rtpchunk?
17:31:52 [dom]
... this could simplify the API
17:32:04 [dom]
Guido: there isn't such a way at the moment, but we could look into one
17:32:50 [dom]
Harald: a constructor with a chunk as input could do this, and avoids touching rtpsenderencodedsource
17:33:24 [dom]
Jan-Ivar: I think it may be a good direction, but I would love to see JS code that shows where the encoded frames are coming from
17:33:30 [dom]
Guido: [shows slide 47]
17:33:44 [dom]
Jan-Ivar: is there a receiver equivalent to this?
17:33:50 [dom]
Guido: that would be the logical follow up to this
17:34:25 [dom]
... e.g. to turn multiple input into a more reliable input for playback
17:35:00 [dom]
RESOLVED: seems like a promising direction for which to see a more complete proposal
17:35:05 [dom]
Topic: -> https://github.com/w3c/webrtc-encoded-transform/pull/215 Keyframe API
17:35:07 [dom]
[slide 54]
17:36:04 [dom]
Harald: #215 is merged - we now have a spec for a keyframe event
17:36:09 [dom]
[slide 55]
17:37:19 [dom]
[slide 56]
17:38:40 [dom]
[slide 57]
17:40:11 [dom]
Jan-Ivar: +1 - hoping we can present API proposals next time around
17:40:38 [dom]
RRSAgent, draft minutes
17:40:40 [RRSAgent]
I have made the request to generate https://www.w3.org/2023/12/12-webrtc-minutes.html dom