15:00:07 <RRSAgent> RRSAgent has joined #webrtc
15:00:07 <RRSAgent> logging to https://www.w3.org/2021/09/20-webrtc-irc
15:00:58 <dom> Present+ Dom, BenWagner, BernardA, Harald, Jan-Ivar, TonyHerre, EladAlon
15:01:04 <dom> Chair: Harald, Bernard, Jan-Ivar
15:02:52 <dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/September_20_2021#WebRTC_WG_Virtual_Interim
15:02:59 <dom> -> https://www.w3.org/2011/04/webrtc/wiki/images/8/86/WEBRTCWG-2021-09-20.pdf Slides
15:03:30 <dom> Topic: Next meetings
15:03:56 <dom> Bernard: October VI to be scheduled 1st week of October - Doodle poll open till nex tweek
15:04:04 <dom> ... then TPAC meetings (joint & solos)
15:05:02 <dom> Present+ ArneSchramm, BrianBaldino, GuidoUrdaneta, TimPanton, YouennFablet
15:06:10 <dom> Topic: Status of recent CfCs
15:06:31 <dom> Bernard: Republishing media capture and streams as CR - completed positively on Sep 17
15:06:33 <dom> Present+ Carine
15:06:44 <dom> ... Jan-Ivar will summarize the chairs decision on it
15:06:57 <dom> ... Another CfC on Transferrable MediaStreamTracks running until Sep 27
15:07:12 <dom> ... our next meeting in October will build on this
15:07:35 <dom> Topic: WHATWG Streams
15:07:47 <dom> Bernard: we have potential dependencies to WHATWG streams
15:08:12 <dom> ... a number of discussions in their repo relate to issues we've discussed in terms of our media processing pipelines
15:08:29 <dom> Topic: Agenda review
15:08:46 <dom> Bernard: main topics: Conditional focus, getViewportMedia, Display surface contraints, echo cancellation
15:09:00 <dom> Topic: Conditional Focus
15:09:47 <dom> Elad: depending on use cases, switching the focus from the browser to the captured window makes more or less sense
15:10:21 <dom> ... focus control is an important part of the user experience, given that making a presentation can be stressful
15:10:50 <dom> ... e.g. if you're capturing a window where you're writing text, focus needs to be there
15:11:11 <dom> ... but there are situations where the browser can be used directly to control to the captured window
15:11:35 <dom> ... the challenge is that the browser cannot determine one situation from another
15:11:55 <dom> ... when the capturing application has a lot more situational awareness
15:12:13 <dom> ... not necessarily complete knowledge, but at least some
15:13:42 <dom> ... I'm proposing an API that associates stream capture with the ability to give a specific limited focus switch opportunity
15:13:56 <dom> ... to the capturing application
15:14:46 <dom> ... because this is done right after the capture is starting (although before a frame is being catpured), the capturing application has all the context it can get to make its decision
15:15:32 <dom> ... the idea is to gives that focus-switching opportunity in a microtask in a promise resolution of the capture request
15:16:10 <dom> ... the proposal includes a number of mitigations (e.g. a 1s timeout) to avoid risks of focus-switching attacks
15:17:15 <dom> ... the particular API I'm proposing is exposed via a method on a subcall of MediaStreamTrack - that way it's only available when obtained through a captured tab or window
15:17:40 <dom> ... we could look at a more finegrained inheritance tree if there is interest
15:18:16 <dom> Jan-Ivar: this is a reasonable problem to solve; I have some concerns with the API surface
15:18:36 <dom> ... since focus switching is global to the user, it doesn't need to be on a mediastreamtrack subclass
15:18:50 <dom> ... it could live e.g. on navigator.mediaDevices
15:19:09 <dom> ... I think a microtask is too narrow - we should queue a task instead, this would give the same presentation
15:19:24 <dom> ... Without having received a frame, how can app determine whether to switch or not?
15:19:43 <dom> Elad: getSettings() on the captured stream can tell you the kind of display surface
15:20:06 <dom> ... checking the content of a frame is likely challenging to get right in any case
15:20:18 <dom> ... looking just at the metadata is easier
15:21:25 <dom> ... re global vs mediastreamtrack, it was partly to protect against attacks based on cloning - but happy to look more into alternatives
15:21:55 <dom> ... task vs microtask - can you say more about your concerns about shim-ability?
15:22:17 <dom> Jan-Ivar: it's a general principle, and I'm not sure the advantages of a microtask in the first place
15:22:48 <dom> Elad: part of it was a concern of backwards compatibility and performance
15:23:15 <dom> Jan-Ivar: I think track & microtask can both address these aspects
15:23:27 <dom> ... in any case, my main concern is where the API lives at the moment
15:23:41 <dom> Youenn: cloning of tracks is known; when you subtype tracks, it starts to be messy
15:23:55 <dom> ... what type would be assigned to a cloned track?
15:24:03 <dom> ... we should avoid subtypes if possible
15:25:50 <dom> ... mitigations of 1s and against busy-looping sound good
15:26:12 <dom> ... I need to think more about the 1s delay
15:26:49 <dom> Harald: re cloning and MST subtracks - we have one case like that, and I think we should change it
15:27:05 <dom> ... we have 2 options: subclassing or making the method returns an error
15:27:15 <dom> ... I don't think JS dev care one way or another
15:27:23 <dom> ... subclassing feels a bit tidier
15:27:43 <dom> Elad: the goal was to reflect our design in the class hierarchy indeed
15:28:26 <dom> Youenn: to get there, I think we should first list the use cases where subtypes actually help - just one method feels not enough to consider changing clone()
15:28:50 <dom> Elad: 3 methods would fit: captureHandler, @@@ only apply to captured media
15:29:41 <dom> Jan-Ivar: I'm opposed to subclassing - I think that API should live in a global space e.g. navigator.mediaDevices.focus
15:29:55 <dom> Harald: where will that written up? I would like to see the argument in more detalis
15:30:00 <dom> s/alis/ails/
15:30:22 <dom> Elad: I'm hearing interest in the API
15:30:31 <dom> Jan-Ivar: interested in solving the problem with a slightly different shape
15:30:57 <dom> Youenn: +1 on a different shape, and discussion on the 1s delay; but sounds like a good space to work on
15:31:58 <dom> [clarification on the 1s requirement makes Youenn happy]
15:32:13 <dom> Topic: getViewportMedia
15:32:38 <dom> -> https://github.com/w3c/mediacapture-screen-share/issues/155  getViewportMedia(): Let pages opt-in to capture #155
15:32:57 <dom> Elad: getViewportMedia is an API allowing to capture the current viewport (what is visible in the tab launching the API call)
15:33:09 <dom> ... equivalent of calling getDisplayMedia and selecting the current tab
15:33:16 <dom> ... there is danger associated with self-capture
15:33:45 <dom> ... to protect against this, we're requiring crossOriginIsolation, opt-in via a header (most likely document policy, but to-be-confirmed)
15:34:05 <dom> ... and only available to top-level docs or priviliged iframes
15:34:10 <dom> s/liged/leged/
15:34:56 <dom> ... Jan-Ivar and I have been discussing a lot and have converged on a number of proposals as summarized in the slide
15:35:12 <dom> Jan-Ivar: we're proposing that getViewportMedia would capture the entire viewport when called from an iframe
15:35:35 <dom> ... and we're proposing using Document Policy with names built on "viewport-capture"
15:37:03 <dom> ... the first proposal is basically deferring the approach to cropping to later
15:37:24 <dom> RESOLVED: getViewportMedia capture the full viewport when called from an iframe
15:38:26 <dom> Harald: re "viewport-capture", is it aligned with the naming convention of Document Policy?
15:39:16 <dom> Tim: just noting the two decisions (iframe capturing the full viewport, and naming) are linked
15:40:56 <dom> RESOLVED: use viewport-capture as naming basis for Document Policy of getViewportMedia
15:41:05 <dom> Harald: these will be confirmed on the mailing list
15:41:23 <dom> Elad: I also intend to suggest a cropping API that might complement getViewMedia in the upcoming months
15:42:23 <dom> Jan-Ivar: getViewportMedia should require user activation
15:42:28 <dom> Dom: +1
15:42:48 <dom> Elad: I can imagine certain cases where use activation makes sense, but others where less so
15:43:40 <dom> ... e.g. if you open a new tab
15:43:58 <dom> Youenn: this feels like a general problem for user activation that is worth discussing in general
15:44:25 <dom> ... but given that this is privileged API, user activation feels like a must
15:47:22 <dom> Dom: +1 on solving it generically for user activation unless we can demonstrate something specific to capturing
15:47:35 <dom> Youenn: note that changing user activation rules is really hard, so we need to get our answer right before shipping
15:48:03 <dom> jan-ivar: removing user activation shouldn't as hard as adding it afterwards
15:48:20 <dom> Elad: I would want more time to make a decision on that particular bit
15:48:27 <dom> Topic: Display surface constraint
15:48:39 <dom> -> https://github.com/w3c/mediacapture-screen-share/issues/184  Revisit: Let getDisplayMedia() influence the default type choice in the picker #184
15:49:03 <dom> Elad: getDisplayMedia doesn't let influence user's choice
15:49:48 <dom> ... user's choice is already being influenced though, by virtue of having a 1st item in the list of choices
15:49:54 <dom> ... Chrome has Screen-first
15:50:08 <dom> ... Safari has only choice (so a major influence)
15:50:14 <dom> ... FF is evolving
15:50:34 <dom> ... Influence could be wielded positively - towards the safer choice, or the more relevant one
15:50:59 <dom> ... a lot of Web developers have expressed interest in allowing influence or limit user's choice:
15:51:11 <dom> ... - save clicks (if the app knows they only want tab, or only want windows)
15:51:25 <dom> ... - apps want to capture audio - only available on a subset of capture sources
15:51:37 <dom> ... - tabs provide higher FPS
15:52:03 <dom> ... - the app knows from context - e.g. allowing to favor slides over other content when doing a presentation
15:52:21 <dom> ... - avoid risk with over sharing
15:53:01 <dom> ... The proposal I'm making is to add a hint as part of the contraints, e.g. "browser: ideal"
15:53:51 <dom> ... the user agent may choose how to apply that hint - from using it to prioritize, to ignoring it or adding warnings in case the UA determines it's not safe to apply the hint
15:54:07 <dom> ... [showing the specific text proposal in #184]
15:54:28 <dom> ... all other contraints are still processed after the user made their choice, only that one gets processed before
15:54:38 <dom> ... it's only a hint, it cannot limit user's choice
15:54:49 <dom> s/browser: ideal/ideal: browser/
15:55:16 <dom> ... e.g. Chrome would show the list of tabs in preference when "browser" is hinted
15:56:04 <dom> Jan-Ivar: in the github discussion, we mentioned additional mitigations - e.g. not listing the requesting tab/window in the list of tabs
15:56:14 <dom> ... would like to see some of these ideas reflected in the text
15:56:35 <dom> ... min & exact constraints are disallowed in gDM, so it would have to be "ideal"
15:56:57 <dom> ... I think it makes sense to use a hint to steer these selectors UI
15:57:22 <dom> ... for clarification, "influence/limiting" requirements discussed earlier were about the app, not the user agent
15:57:56 <dom> Harald: re removing the calling tab, would it be only for this usage of the hint, or any use of gDM?
15:58:12 <dom> Jan-Ivar: I think they need to be considered before we add this
15:58:37 <dom> Elad: my recollection was we would encourage the UA to warn of risks of self-capture rather than removing the option altogether
15:59:34 <dom> ... there are other ways of adding friction that doesn't require removing the option completely
15:59:52 <dom> ... removing it completely might create risks of oversharing via sharing of the entire screen
16:00:28 <dom> Jan-Ivar: I think we can probably converge on mitigations for self-capture
16:00:50 <dom> ... ideally, I would like normative language
16:01:10 <dom> Present+ SergioMurillo
16:01:27 <dom> Youenn: should we allow a hint for capturing the entire screen? that's the riskiest
16:01:36 <dom> ... let's focus on hinting towards capturing less
16:02:20 <dom> ... In general, I dislike constraints - can we add a dedicated parameter instead of reusing the contraints syntax?
16:02:44 <dom> ... this may open further extensibility down the line (e.g. highlight tabs from a given origin?)
16:03:12 <dom> ... can you share more about Chrome's plans in terms of mitigations against self-capture and its dangers?
16:03:38 <dom> Elad: we haven't prototyped the warning mechanism yet
16:03:52 <dom> ... re constraints, I have no objection to using a parameter instead of constraints
16:04:25 <dom> ... re removing "screen" - it's interesting, but if that is the default when no hint is given, this isn't really helping
16:04:37 <dom> Youenn: that default behavior is specific to Chrome
16:05:08 <dom> ... Safari only allows screen, but we will have a picker at some point where screen won't be the default
16:05:22 <dom> ... and I don't think apps should have a way to default to screen
16:05:43 <dom> Jan-Ivar: FF already doesn't default to screen, and +1 to youenn of not allowing (or just ignoring) screen as a constraint
16:06:07 <dom> Elad: the user agent would already be free to ignore the hint
16:06:42 <dom> ... for Chromium, getting visibility on dev's intent would be useful in migrating away from that default
16:07:16 <dom> Present+ SongXu
16:07:27 <dom> Present+ ThomasGuilbert
16:08:01 <dom> Bernard: in terms of the requests from developers, is audio capture only avaiable on screen?
16:08:13 <dom> Elad: no, it's available on tab, and screen on windows
16:09:34 <dom> Bernard: re high-FPS capture - is that typically tab?
16:09:39 <dom> Elad: in Chromium, yes
16:10:04 <dom> ... but it's in general, a way for developers to steer toward what they know will work for their use cases
16:10:20 <dom> Bernard: is "screen"-level capturing key to any of these requests?
16:11:12 <dom> Elad: right; but note that "screen" could be used to capture from a different monitor
16:11:22 <dom> Jan-Ivar: but all monitors are dangerous
16:11:54 <dom> Elad: so I'm hearing support except for the the screen-hint
16:12:46 <dom> TimP: I dislike heuristics-based picker - it makes it a nightmare to test and makes everything unpredictable
16:13:05 <dom> Elad: the mention for heuristics was for apps to use, not the UA
16:14:24 <dom> Jan-Ivar: supporting, but with stronger language on warnings for self-capture
16:14:41 <dom> Topic: Echo Cancellation
16:15:00 <dom> -> https://github.com/w3c/mediacapture-extensions/issues/31  Echo cancellation: Need to specify the source of the echo cancellation reference signal #31
16:15:11 <dom> -> https://github.com/w3c/mediacapture-extensions/pull/32  Specify constraint echoCancellationReferenceSinkId #32
16:15:22 <dom> Harald: this is a request coming from our audio team
16:16:15 <dom> ... echo cancellation is about removing the audio picked up by the microphone in the room to keep only the audio generated *in* the room
16:16:31 <dom> ... it's in general complicated - a complicated part is knowing what to remove
16:17:02 <dom> ... current implementation in Chrome just looks at what's coming it via the peerconnection
16:17:22 <dom> ... this has proven insufficient and we want to revise this
16:18:20 <dom> ... if we want to remove audio output, you can hit issues with specific headphones or setups
16:18:45 <dom> ... from the application perspective, you want to identify what output has been used that is most relevant to echo cancellation and feed that to the algorithm
16:19:03 <dom> ... to keep it simple, we have an enumaration of output devices via sinkIds
16:19:39 <dom> ... the proposal is to re-use this sinkid in the contraint for echo cancellation
16:20:47 <dom> TimP: +1 to do something in this space
16:21:02 <dom> ... will it help if you mix WebAudio in?
16:21:22 <dom> ... i.e. when the audio output comes from WebAudio processing
16:21:44 <dom> Harald: yes, it should cover this (as long as the output makes it to the speaker)
16:22:00 <dom> Jan-Ivar: Mozilla doesn't believe this API is needed to do correct echo cancellation
16:22:25 <dom> ... why does the UA needs JS input on this? The UA already know which headset is being used
16:23:05 <dom> ... it's not clear what getting input from the app is useful here
16:23:28 <dom> Harald: which audio output is currently used by the echo cancellation?
16:23:40 <dom> Jan-Ivar: I believe we have access to the rendered output (incl out of WebAudio)
16:23:53 <dom> ... Paul Adenot is our key person on this
16:24:14 <dom> Harald: would like his opinion on the headcase
16:24:32 <dom> Youenn: +1 to Jan-ivar - the UA should already have access to the all info it needs
16:24:57 <dom> ... and it has more info that apps would have on this
16:25:53 <dom> bernard: Harald, you said chrome currently uses sum of all audio outputs from peerconnection
16:26:09 <dom> ... is the intent here to improve the chromium implementation or to let them do better echo cancellation?
16:26:23 <dom> harald: this is not for app-based echo cancellation
16:27:25 <dom> bernard: I've heard requests from apps to do have an adjustable echo cancellation - e.g. an echo cancellation transform stream
16:27:45 <dom> Harald: that is orthogonal to this proposal
16:28:00 <dom> ... echo cancellation can't be modeled as a transform stream: it's a 2 input objects
16:28:11 <dom> ... it can be modeled as process that takes 2 audio inputs
16:28:28 <dom> youenn: you could still do 1 input / 1 output with an additional parameter
16:29:03 <dom> ... in the transform stream creation with the reference stream
16:29:22 <dom> Harald: interesting thing to do, but not this proposal
16:30:07 <dom> TimP: there are situations where you don't want to cancel part of the stream being output - e.g. background music
16:30:20 <dom> ... with the room accoustics
16:30:52 <dom> ... maybe a rare use case, but one we've stumbled upon it for immersiveness
16:31:00 <dom> harald: you could turn echo cancellation off?
16:31:07 <dom> timP: but that generates other issues
16:32:34 <dom> Sergio: I don't think this proposal would help solve the Chrome issue
16:33:48 <dom> ... there are 3 different issues being discussed: echo cancellation in Chrome, new echo cancellation tuning use cases (that would need clarification/refinement), and exposing echo cancellation separately from WebRTC (maybe in Web Audio)
16:34:11 <dom> Harald: I'm hearing opposition to making an API of the specific proposal because the UA should be able to figure it out
16:35:10 <dom> ... I find it interesting that only browser output should be cancelled - if you have another app than the browser producing audio, shouldn't it be removed too?
16:35:44 <dom> Jan-Ivar: RNNoise has been exploring some of this; but echoCancellation: true is likely focused on the meeting use case
16:36:00 <dom> Youenn: the OS can also provide user-configurable echo cancellation styles
16:36:56 <dom> Guido: the motivation for Chrome is to help figure which of the output devices should be used as the reference signal for echo cancellation
16:37:22 <dom> ... if there are several audio output devices with one being preferred by the app
16:37:52 <dom> Harald: I'd like to invite comments on the issue on whether this API is needed or not
16:38:00 <dom> ... I haven't seen much comments on the shape of the API
16:38:20 <dom> ... if we were to conclude there was such a need, this API may be OK
16:38:26 <dom> ... but no consensus on the need for such an API
16:39:46 <dom> Topic: Wrapping up
16:40:14 <dom> Bernard: any CfC needed based on our discussions?
16:40:31 <dom> Jan-Ivar: re getViewportMedia, should we put this in a new doc or an existing one?
16:43:08 <dom> Dom: having a single document couple their process progress
16:43:22 <dom> elad: also keeping them separate helps making clear how distinct they are
16:43:37 <dom> youenn: it also helps in terms of separating the test cases in different folders
16:45:09 <dom> harald: sounds like convergence towards a separate spec
16:45:14 <dom> jan-ivar: would still prefer a single doc
16:45:27 <dom> Topic: October meeting
16:45:51 <dom> Bernard: next meeting will be devoted to mediacapture-transform - proposed content and agenda was shared on the list
16:46:09 <dom> -> https://lists.w3.org/Archives/Public/public-webrtc/2021Sep/0030.html Preview of October Virtual Interim slide deck
16:47:47 <dom> ... there is overlap between mediacapture-transform and WHATWG streams issues
16:48:11 <dom> Youenn: I will try to mark more explicitly issues in MC-T that are linked to WHATWG streams
16:50:20 <dom> Bernard: part of what I thought might be useful to hear is where these upstream WHATWG stream issues are on the roadmap (if at all)
16:50:51 <dom> Jan-Ivar: the new proposal we want to present is streams-based, but improvements over the existing one
16:50:57 <dom> ... still needs some fixes in WHATWG streams
16:52:32 <dom> ... I have linked demos in the slides for some of the issues we're trying to address
16:54:10 <dom> TimP: it would be good to start these presentations with use cases to scope our discussions
16:54:39 <dom> Jan-Ivar: the slides Youenn and I developed includes goals of the proposals
16:54:55 <dom> Harald: Media Capture Transform starts with use cases
16:55:09 <dom> Bernard: Streams have been adopted to use streams to manage pipelines
17:00:22 <dom> Youenn: please send early feedback on the proposals
17:00:33 <dom> RRSAgent, draft minutes
17:00:33 <RRSAgent> I have made the request to generate https://www.w3.org/2021/09/20-webrtc-minutes.html dom
17:00:41 <dom> RRSAgent, make log public
17:04:53 <dom> RRSAgent, draft minutes
17:04:53 <RRSAgent> I have made the request to generate https://www.w3.org/2021/09/20-webrtc-minutes.html dom
17:05:44 <dom> Meeting: WebRTC September 2021 virtual interim
17:05:56 <dom> RRSAgent, draft minutes
17:05:56 <RRSAgent> I have made the request to generate https://www.w3.org/2021/09/20-webrtc-minutes.html dom