15:58:41 <RRSAgent> RRSAgent has joined #webrtc
15:58:45 <RRSAgent> logging to https://www.w3.org/2024/12/10-webrtc-irc
15:58:45 <Zakim> Zakim has joined #webrtc
15:58:45 <dom> RRSAgent, make log public
15:58:51 <dom> Meeting: WebRTC December 2024 meeting
15:58:51 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/November_19_2024
15:58:51 <dom> Slideset: https://docs.google.com/presentation/d/1wk-0WiD1aTqcQUYDiNJcP-XQ421KCbY6A1xX_-MKy_A/edit#slide=id.g2bb12bc23cb_0_0
15:58:51 <dom> Chairs: Guido, Jan-Ivar, Bernard
15:59:15 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/December_10_2024
16:01:58 <dom> Present+ Jan-Ivar, Guido, Harald, Tove, Peter, Bernard, TimP
16:02:33 <dom> Recording is starting
16:03:25 <dom> Present+ Carine
16:04:23 <dom> Topic: Captured Surface Switching
16:04:23 <dom> [slide 10]
16:05:36 <dom> Subtopic: -> https://github.com/w3c/mediacapture-screen-share-extensions/issues/15 Auto-pause
16:05:36 <dom> [slide 12]
16:05:58 <dom> Present+ Youenn
16:06:53 <dom> [slide 13]
16:07:40 <dom> [slide 14]
16:08:16 <dom> [slide 15]
16:09:06 <dom> [slide 16]
16:10:13 <dom> [slide 17]
16:12:09 <dom> [slide 18]
16:12:35 <dom> [slide 19]
16:13:56 <dom> Youenn: re slide 17, let's take an example of a Web app wanting to pause to ask the user to confirm its intent to share the content
16:14:21 <dom> ... you want to pause sending to remote parties, but want to keep the local view of the shared content
16:14:39 <dom> ... you need different behaviors for the two
16:15:52 <dom> Tove: with the callback, you can do the same thing as with the event
16:16:07 <dom> Youenn: but that shows that freezing the video would be hack-ish in both situations
16:16:29 <dom> ... in a PC, you can use setParameter or replaceTrack() to handle this
16:16:46 <dom> ... if you're in a call with a local preview with a track in the worker sent to the remote party
16:16:59 <dom> ... you want to stop sending the track in the worker
16:17:39 <dom> ... the guarantee you get from the event makes it a better API; you may not even need to pause the track
16:18:26 <dom> ... I prefer the event, would be OK with adding the callback although not sure about having it async, although maybe the cropTarget makes it compelling
16:18:43 <dom> ... but would want to understand it better
16:18:58 <dom> ... otherwise would prefer a sync callback
16:19:31 <dom> ... in terms of ordering, it would be better to first fire the event (ensuring the settings are updated) and then the callback can access accurate information
16:20:17 <dom> Jan-Ivar: the UI use case can be solved with either API; the only need for async would be for replaceTrack or cropTo
16:20:47 <dom> ... configurationchange exists today, mostly to adapt consumer of a track to its properties, with throttling to avoid correlating across origins
16:20:54 <dom> ... proposal 1 feels simpler
16:21:30 <dom> ... also deviceId is not currently specified at the moment, so wouldn't be exposed via configurationchange at the moment
16:22:04 <dom> ... also the event may not be clear about their causes which might create unexpected bugs for developers who wouldn't handle all situations
16:22:30 <dom> Tove: the configurationchange event is only for a few cases today
16:22:48 <dom> Youenn: the event should be fired whenever the settings are changed without involvement from the application
16:23:04 <dom> ... e.g. when the user agent or the OS enables blur - the app should be able to keep track of that
16:23:07 <dom> Present+ SunShin
16:23:22 <dom> ... this matches what happens when a surface changes (outside of the Web app control)
16:23:33 <dom> ... re fuzzing, I don't think it's useful in this context - we should remove the related note
16:24:05 <dom> ... when you're processing in the worker, having the callback in `window` make it painful since you need to `postMessage` back and forth
16:24:28 <dom> ... if we agree with `deviceId` and ordering, it seems cheap to support the `configurationchange`
16:24:40 <dom> Tove: so you're supporting both approach?
16:25:17 <dom> Youenn: I'm ok with it - but still unsure about the callback being async (need to discuss more the cropTarget use case)
16:25:53 <dom> ... I'm reluctant to adding a new state where frames aren't emitted outside of `track.enabled` - but would consider it if necessary
16:26:08 <dom> Jan-Ivar: at the moment, screen capture tracks don't expose deviceIds
16:26:36 <dom> ... if we decide later today to expose it, this would require firing `configurationchange`
16:26:46 <dom> ... we're set to discuss it later on this agenda
16:27:14 <dom> Youenn: `deviceId` would be a small addition that would help with detecting surface change - we will indeed discuss it later
16:29:31 <dom> Tove: so if we conclude to adding it, we would go with both?
16:29:50 <dom> Youenn: if we do need an async callback, having just the callback would be fine
16:30:25 <dom> Jan-Ivar: I'm not fond of having both as they feel redundant
16:33:28 <dom> Youenn: having both (if we go in the right order) might queue two or three tasks when dealing with multiple tracks, but the resulting delay shouldn't be meaningful
16:33:54 <dom> ... (this doesn't apply in the worker case since no synchronization is needed in that case)
16:34:15 <dom> ... is there agreement about firing the event before the callback or is that too early to decide?
16:35:38 <dom> ... today, you already get an event when switching from screen to window
16:36:06 <dom> Tove: the spec isn't very clear on the circumstances of when the configurationchange event should fire
16:36:21 <dom> Jan-Ivar: I think the algorithm refers explicitly capabilities and settings
16:36:56 <dom> Subtopic: -> https://github.com/w3c/mediacapture-screen-share/issues/308 [screen-share] Issue 308: Should screen capture tracks expose deviceId?
16:36:58 <dom> [slide 33]
16:40:14 <dom> Youenn: not huge use case, but low complexity and no specific drawback that I could identify
16:40:37 <dom> jan-ivar: my main concern is that `deviceId` doesn't match how it is used in other specs
16:41:49 <dom> Youenn: `deviceId` is identifying the source
16:42:32 <dom> jan-ivar: the chapterizing use case would be better done with labels from my perspective; it makes assumptions about how one would go about this
16:43:16 <dom> ... I think that leaves configurationchange as the main use case
16:44:05 <dom> Guido: the fact that `deviceId` doesn't change for a camera doesn't invalidate the idea of using for a possibly changing source
16:44:28 <dom> ... I like the idea of exposing it and signaling its change with configurationchange
16:45:45 <dom> dom: exposing `deviceId` sounds reasonable to me to
16:46:05 <dom> Jan-Ivar: ok, since I don't have a strong reason to object to it, I think we probably have consensus on adding it
16:46:44 <dom> RESOLVED: Consensus to add `deviceId` to settings of a track
16:47:03 <dom> Youenn: I'll check how this impact to canvas sourced tracks (and how they behave today)
16:47:26 <dom> Subtopic: https://github.com/w3c/mediacapture-screen-share-extensions/issues/15 Back to Auto-pause
16:47:53 <dom> Tove: so Youenn you asked about the async callback use case with regard to cropTarget?
16:48:32 <dom> Youenn: is the use case that you're cropTargetting in a VC, you switch surface and then you want to pause both local/remote tracks before deciding whether to re-crop?
16:49:22 <dom> Tove: imagine switching to a tab with slides and speaker notes, you'll want to cropTo to resolve before sending frames with the additional content
16:50:27 <dom> Youenn: in that use case, there is no local preview? if so, why not use setting `track.enabled` to false or stopping it on the PC?
16:51:14 <dom> Tove: there may be use cases where this would also apply to a local track - keeping the two tracks in sync as part of the callback feels cleaner
16:51:58 <dom> Jan-Ivar: I'm a fan of simplicity - if we have the configurationchange event, I would prefer to only have the event API, not also the callback
16:53:49 <dom> Guido: I think we can start with `deviceId` and if additional use cases show that having both would be beneficial, we can revisit it
16:54:25 <dom> RESOLVED: proceed with configurationchange event as the way to manage surface switching handling
16:55:05 <dom> Subtopic: -> https://github.com/w3c/mediacapture-screen-share-extensions/issues/16 Cross-type surface switching
16:55:05 <dom> [slide 20]
16:56:53 <dom> Tove: this would be a hint more than a hardset requirement
16:58:06 <dom> Jan-Ivar: what is the use case for not allowing surface switching, beyond backwards compat?
16:58:13 <dom> Tove: it's only for backwards compat
16:58:35 <dom> Jan-Ivar: if so, I would rather not add an API just for that
17:00:11 <dom> Tove: if an app expects only a tab, the app may not expect they need to update their UI (e.g. cropTarget no longer is possible)
17:00:30 <dom> Jan-Ivar: cropTo would fail, which is probably what you want
17:00:52 <dom> Youenn: I would look at what OS do; in macOS, it's the OS controlling the UI to pick surfaces (not the UA)
17:01:43 <dom> ... I haven't checked if there is an option to do that on macOS - if it's not doable at the OS level, then it won't be implementable there, in which case I would be reluctant
17:01:57 <dom> Tove: this is exposed on the macOS API
17:02:32 <dom> Youenn: OK - if it's implementable, and if native apps find it useful, that suggests this is a useful API to consider
17:02:58 <dom> ... it would definitely need to be hint, and make it possible for the UA/user to override
17:03:04 <dom> ... I would use include as the default
17:03:39 <dom> Tove: I'm hearing stronger examples needed for "exclude"
17:04:17 <dom> Jan-Ivar: if it's a hint, maybe backwards compat doesn't need to be standardized
17:04:33 <dom> Harald: we had a similar situation with the plan B transition
17:05:32 <dom> Harald: so the conclusion is that cross-type surface switching is always on?
17:05:56 <dom> Youenn: I'll look into the use cases that supported the macOS API and whether they justify a Web API
17:06:11 <dom> Topic: Timing info for Encoded Frames
17:06:11 <dom> [slide 23]
17:07:13 <dom> Topic: Timing info for Encoded Frames
17:07:13 <dom> [slide 24]
17:08:54 <dom> [`captureTimestamp` in the slide should be `captureTime`]
17:09:02 <dom> Topic: Timing info for Encoded Frames
17:09:02 <dom> [slide 25]
17:09:39 <dom> Youenn: maybe we could use a single dictionary to have a single definition
17:09:56 <dom> Guido: any opposition to add receiveTime to encoded frames?
17:10:24 <dom> ... receiveTime would be defined as the timestamp of the last packet necessary to build the video frame once received
17:11:35 <dom> Jan-Ivar: are there cases where the said frame wouldn't be transmitted?
17:12:20 <dom> ... e.g. with WebCodecs + RTCEncodedSource?
17:12:27 <dom> Guido: the time it reaches the receiver
17:12:41 <dom> Harald: things that aren't received shouldn't have a receiveTime
17:12:57 <dom> Bernard: this is a good proposal, and I like having it both in WebCodecs and in EncodedFrame
17:13:18 <dom> Youenn: not sure I'm convinced with WebCodecs, but will discuss on github
17:13:36 <dom> RESOLVED: proceed with adding receiveTime to Encoded* Metadata
17:14:07 <dom> Topic: -> https://github.com/w3c/webrtc-pc/issues/3014 [webrtc] Issue 3014: Spec says to send black frames for ended tracks
17:14:12 <dom> [slide 29)
17:14:17 <dom> s/9)/9]
17:17:09 <dom> Youenn: I would align the spec with what implementations are doing; it may not be the best, but it's the current status and apps seem fine with it
17:17:47 <dom> ... it would be good to create consistency for the last point - maybe raise a separate issue
17:18:09 <dom> RESOLVED: Proceed with aligning with current implementations
17:18:26 <dom> Topic: -> https://github.com/w3c/mediacapture-extensions/issues/158 [mediacapture-extensions] Reduce scope of MediaStreamTrack transfer to DedicatedWorker for now
17:18:29 <dom> [slide 30]
17:19:54 <dom> Youenn: the VideoFrame object is only exposed in Window and DedicatedWorker, likewise for the MSTP
17:20:03 <dom> ... it makes sense to reduce this for DedicatedWorker
17:20:16 <dom> ... Safari is only implementing transfer to DedicatedWorker in the same cluster
17:20:24 <dom> ... (same for VideoFrame)
17:20:37 <dom> ... not sure if that needs to be spelt out
17:20:56 <dom> Jan-Ivar: we could look into that and if there are precedents for restricting to a cluster
17:21:15 <dom> RESOLVED: Proceed with reducing scope of MediaStremaTrack to DedicatedWorker
17:21:26 <dom> Topic: -> https://github.com/w3c/mediacapture-transform/issues/115 Issue 115: What is the expected timing of MSTP video frame enqueuing with other track events
17:21:30 <dom> [slide 31]
17:23:12 <dom> [slide 32]
17:26:01 <dom> Bernard: regarding the last one, I wouldn't try synchronizing getSettings and videoframe properties
17:26:16 <dom> Youenn: that would require exposing all the settings in the video frame
17:26:40 <dom> Bernard: maybe not all - e.g. facingMode wouldn't be useful; but some we're already discussing exposing (e.g. blur, width and height)
17:26:57 <dom> Jan-Ivar: 2 questions here: timing, and what to expose in videoframe metadata
17:27:02 <dom> ... focusing on timing for now
17:27:28 <dom> ... it's good to specify things, but I'm not sure how many synchronous guarantees we can provide given the buffering
17:27:44 <dom> ... coupling things too tightly might make it challenging to add transform
17:28:26 <dom> ... the buffer may lead us to handle different e.g. the mute / unmute event
17:28:43 <dom> Youenn: I'm talking specifically about enqueuing frames, not reading frames from the stream
17:29:30 <dom> ... because we're enqueuing a task, this provides an opportunity to synchronize things
17:30:00 <dom> ... if we specify unmute, it seems logical to specify the mute situation as well, symetrically
17:30:42 <dom> ... I agree with Bernard's point about getSettings; I'm not sure how to handle applyConstraints or configurationChange
17:31:06 <dom> Jan-Ivar: how would handle buffers, e.g. maxBufferSize = 10?
17:31:15 <dom> Youenn: this shouldn't have any impact
17:31:37 <dom> Guido: this should be dealt at the sink level (where buffering happens)
17:32:23 <dom> ... not sure it needs to have something specific for MSTP - it mostly deals with things outside the track
17:33:02 <dom> Youenn: one model we could consider: mute/unmute events are always sent in the same thread; we enqueue tasks for the unmute event
17:33:20 <dom> ... for the rest, we consider sources single-threaded
17:33:41 <dom> Harald: VideoFrames are observable on an HTML video element
17:34:15 <dom> ... if we get a mute event, and we get a frame that was sent after the mute event, that's clearly an error - we should prevent it
17:35:29 <dom> ... we can say something like the events have to act as if they were ordered, as if they came from the same event source
17:35:48 <dom> Jan-Ivar: I would like to understand better as to why this is problem
17:36:01 <dom> ... focusing on the event might the wrong approach
17:36:14 <dom> ... e.g. muting happens in the background, and is only surfaced in JS later
17:36:40 <dom> ... you could probably write a WPT with VideoTrackGenerator
17:36:57 <dom> ... my only worry is to make things too synchronous
17:37:24 <dom> ... I'm not sure there is an overall solution, we should look at each case
17:37:40 <dom> Youenn: agree about not dealing with getSettings and videoframes
17:37:54 <dom> ... I can try and detail more situations on github
17:38:03 <dom> Jan-Ivar: shouldn't it apply to track.enabled as well?
17:38:08 <dom> Youenn: I should look into it
17:38:31 <dom> Bernard: re configurationchange and applyConstraints - are we clear on the effect of these events on videoframe properties?
17:38:42 <dom> ... otherwise, I agree with Jan-Ivar on the risk of being too prescriptive
17:40:02 <dom> ... we shouldn't make this based on the timing of the events, but instead base on videoframe properties
17:41:21 <dom> Youenn: ok, so more discussion is need on github
17:41:28 <dom> Topic: -> https://github.com/w3c/mediacapture-output/issues/147 [mediacapture-output] Issue 147: Implicit consent via getUserMedia should allow access to non-miked speakers
17:41:36 <dom> [slide 35]
17:43:06 <dom> [slide 36]
17:45:17 <dom> Jan-Ivar: my concern is that this might be blocking on https://github.com/w3c/mediacapture-main/issues/1019
17:45:38 <dom> ... not sure we should proceed with this before we get clarity on that one
17:46:00 <dom> ... if we maintain the existing spec requirement, this might make sense
17:47:17 <dom> Youenn: we could disentable the two by noting you need microphone access for exposing speakers (but we should make progress on the other one)
17:47:38 <dom> Guido: https://github.com/w3c/mediacapture-main/issues/1019 is orthogonal to this one since we're already exposing speakers - the discussion is which speakers we expose
17:47:56 <dom> ... when is enumeration allowed is orthogonal
17:48:20 <dom> Youenn: does Chrome expose speakers without capture?
17:48:29 <dom> Guido: it gates enumeration on permission
17:48:45 <dom> Youenn: would you be able to align with the spec for speakers?
17:48:54 <dom> Guido: the problem is Web compatibility
17:49:30 <dom> ... implementing the spec was not web compatible, so we had to rollback hence https://github.com/w3c/mediacapture-main/issues/1019
17:51:05 <dom> Jan-Ivar: the 2 situations are connected since the decision on this may lead to different implementation across browsers
17:51:26 <dom> Bernard: I agree with Guido they're orthogonal
17:51:47 <dom> ... would like to support exposing all speakers
17:51:58 <dom> Jan-Ivar: I would object with proceeding before https://github.com/w3c/mediacapture-main/issues/1019 is resolved
17:52:08 <dom> Youenn: let's try to discuss it at the next meeting then
17:52:52 <dom> RRSAgent, draft minutes
17:52:53 <RRSAgent> I have made the request to generate https://www.w3.org/2024/12/10-webrtc-minutes.html dom
18:55:04 <Zakim> Zakim has left #webrtc