WebRTC May 2022 VI

17 May 2022


Bernard, Byron_Campen, Carine, Dom, Eero, Elad, Florent, Jan-Ivar, Patrick, Riju, Tuukka_Toivonen, Youenn
Bernard, Jan-Ivar

Meeting minutes

Recording: https://www.youtube.com/watch?v=wlwi7491ERo

Slideset: https://lists.w3.org/Archives/Public/www-archive/2022May/att-0000/WEBRTCWG-2022-05-17.pdf

Future meetings 🎞︎

Bernard: OK with moving up our next meeting to June 7th?

WebRTC 🎞︎

RTP Header Extension Encryption 🎞︎

RTP Header Extensions Encryption (cryptex) #106

[Slide 12]

Bernard: cryptex has completed IETF last call, with Sergio incorporating changes from comments

[slide 13] an API proposal from Sep 2020 meeting
… with RTP headers being encrypted as all or nothing per bundle
… with a policy on whether this encryption is negotiable or required

[Slide 14]

RTP Header Extensions Encryption (cryptex) #106

Bernard: please review the PR
… that adds this API
… a little complicated since it needs change patching the webrtc-pc steps

[Slide 15]

Jan-Ivar: any reason it's not a readonly getParameters

Bernard: it is
… it's set per transceiver (although it's really per bundle)

youenn: lgtm
… could we simplify by not doing it per transceiver?

bernard: how would you handle the situation where you receive an offer with different setting on different bundles?
… bundles don't get surfaced in the API

Byron: it could be a property on the transport, which is what represents the bundle

Bernard: please review the PR and bring your comments there

Jan-Ivar: what happens if this gets changed with setConfiguration?

Byron: really a bad idea :)

Bernard: should we prevent that from happening?

Byron: +1
… changing that is really awful, and so would writing tests for it
… let's keep it simple in absence of a use case for it

Issue #2735: webrtc-pc does not specify what level of support is required for RFC 7728 (RTP pause) 🎞︎

[Slide 18]

Bernard: this is about confusing about RFC 7728 (rtp stream pause) and the level of required support for it
… this isn't required

Byron: there is one place in the spec that refers to it in the API spec
… supporting the ~ flag requires support for 7728
… there seems to be support towards instead removing support for that flag

[Slide 19]

Bernard: not even sure why we would want to support it in the SDP

Bernard: I think it shouldn't be there; let's make this as ready for PR and figure out the right pull request to fix this

CaptureController 🎞︎

[Slide 32]

#190 Conditional Focus [screen-share]

[Slide 33]

Jan-Ivar: a proposal that helps addressing 3 issues, starting with https://github.com/w3c/mediacapture-screen-share/issues/190

#57 .sendCaptureAction() misplaced [capture-handle]

[Slide 34]

#12 .getCaptureHandle() misplaced? [capture-handle]

[Slide 35]

[Slide 36]

Jan-Ivar: similar to Fetch's AbortController
… this can give more leeway on the acceptable time to give focus

Elad: the controller patterns looks nice
… would you be open to exposing a getter of the controller on the track?
… if an API returns several streams, would it return several controllers?

jan-ivar: on the first question, not exposing it on the track would be a feature
… adding a getter could be easily done by apps any way

elad: makes sense

jan-ivar: I haven't thought about multi-capture yet - happy to look into it; I would expect one controller per capture

elad: different type of capture (e.g. tab, window, screen) may require different type of controller
… e.g. focus on a screen doesn't make a lot of sense

jan-ivar: for "monitor", focus() would resolve immediately - essentially a no-op

elad: with subclassing, we could determine whether there is a focus() method - screen-subclass or audio-tracks wouldn't have it

jan-ivar: that's worth discussing - subclassing feels a bit excessive at this point, but to be discussed

youenn: separating functionality specific to a source from the track is a good idea
… adding it to track isn't a great model
… I'm less sure about the API surface, but we can iterate on it
… focus() is one thing; supported actions is a different thing
… they may require different objects
… e.g. actions may benefit from being transferable
… whereas focus handling doesn't really make sense

elad: how quickly do we think we can agree on an API shape?
… I wouldn't want this to delay other long-started discussions e.g. on conditional focus

Jan-Ivar: not big fan of subclassing; making the type of capture source detectable would be more useful

dom: maybe let's focus on using this pattern for conditional focus

elad: how do we deal with feature detection?

youenn: through the presence of the interface, as for other APIs

elad: not necessarily convinced, but +1 if it helps moving faster

dom: should this be discussed as part of screne-share repo then

jan-ivar: I'll create an issue in screen-share to help discuss the API shape before we move to a PR

WebRTC Encoded Transform 🎞︎

PR #132

[Slide 39]

youenn: we agreed to add a generateKeyFrame API
… adding the frame timestamp would be a good addition, but this may result in having to have multiple timestamps when dealing with multiple RIDs

[Slide 40]

youenn: proposing instead to make the method apply to a single rid

aboba: lgtm

youenn: ok, so will finalize the PR

Extensions to Media Capture and Streams 🎞︎

PR #61: Add support for background blur and configuration change event 🎞︎

[Slide 41]

youenn: lots of background blur done with canvas and WebGL today
… OSes are now adding support for background blur
… the intel team has made measurements showing that OS background blurs gives a 2x power improvement over JS-based blur
… also, if the OS is already providing it, makes no sense to add another layer of blurring

[Slide 42]

youenn: proposal is to add a backgroundBlur constraint, a capability & a setting
… this would be a boolean constraint, as a starting point
… a double [0,1] range may be worth discussing, but I suggest starting with a boolean approach
… interested in feedback in the approach, and on the boolean vs double discussion?

jan-ivar: lgtm

riju: would shifting to double later be an option?

youenn: not sure about the migration path; it could be a new constraint which would work on a clear definition
… hearing support, so we can finalize and merge the PR then

[Slide 43]

youenn: the same PR #61 also includes a configuration change event
… since the OS setting can change outside of the UA control

dom: how broadly would this event apply? any constraints? a subset?

youenn: any track setting

jan-ivar: how would handle "exact" on a backgroundBlur applyConstraint?

youenn: if set at the OS level, it would reject

jan-ivar: ... but that only applies to "exact" constraint

youenn: right

jan-ivar: a UA could implement this and provides this per-track?

youenn: good question to discuss
… on iOS if you disable echoCancellation, it can't be disabled for a single track - so there are precedents

jan-ivar: apps can turn this on & off with applyConstraints - is that OK?

youenn: the browser could keep control on that within the constraints framework

florent: which capabilities would trigger configuration change?
… you mentioned OS level ones
… what if a device is open in several browsers and one would change e.g. the exposure?

youenn: we usually try to hide this - so the configuration change event will be at the discretion of the UA
… if two pages are capturing, the UA is in control
… but we need to be careful about broadcasting events across origins

jan-ivar: [on the chat] similar to devicechange we should add fuzzing

florent: +1 otherwise it could be used as a communication channel

youenn: florent, could you comment in that direction on the PR?
… hearing consensus in general on PR #61

jan-ivar: rough agreement at least :)

PR #59: Add powerEfficientPixelFormat constraint 🎞︎

[Slide 43]

youenn: submitted by henrik
… some cameras generation MJPEG video frames, which can have a negative impact on power consumption
… proposal is to add a new constraint to help

jan-ivar: sounds a good approach

dom: when would you not want this?

youenn: for specific resolution capture, or e.g . for a single frame capture
… the issue is that it's hard to have a good default with backwards compat
… there is leeway for the UA to pick an efficient camera

dom: but then shouldn't this be the UA doing the right thing rather than asking all developers to update their code?

jan-ivar: UAs might have different defaults on desktop vs mobile
… with exposing the info is getSettings, you can decide to make different compromise in terms of resolution vs power consumption

dom: getSettings I understand; device selection feels icky

youenn: it wouldn't be allowed as a required constraint in getUserMedia

jan-ivar: +1 to that - we don't want to exclusde people that only have MJPEG cameras

dom: hearing we should be doing this

Dynamic Source for Screensharing 🎞︎

[Slide 45]

[Slide 46]

[Slide 47]

[Slide 48]

[Slide 49]

Elad: apps may not expect the content of the source to be changing under their feet; e.g. in the context of a capture handle

[Slide 50]

[Slide 51]

[Slide 52]

elad: when the browser provides a feature that apps can't predict or control, this creates a bad experience for app developers (and ultimately end users)

[Slide 53]

youenn: I'm a bit surprised by the API surface; I'll need thinking some more, maybe the UA could do some heuristics
… not sure an app can determine whether it wants to support this at the time of getDisplayMedia
… have you discussed a more flexible API?
… or is it hyperfocused on purpose?

elad: I'm up for giving more flexibility to apps
… re heuristics, they're not going to serve well all apps equally

youenn: what would be the default?

elad: I would argue that backwards compat should default to the existing behavior (not dynamic sources)
… but open to discuss this

jan-ivar: a bit concerned that it would let the application limit the user choice
… I like the feature in Chrome, dynamic switching is a good feature
… also questions when source changes whether focus should be set
… not sure if we shoud allow any app decide whether or not to allow the user to switch
… how much of the use case is tied to the selfCapture constraint
… re capture handle, capturing a different tab is not much different from navigating from a captured tab

elad: re allowing limiting user selection - right now, there is no choice offered to the user, this would add a new choice
… re focus, with the current Chrome implementation of the UX, you can only press the button when the tab is focused, so it's not an issue for us at the moment; may need discussions if that changes
… re self-capture, you're right that's the clearest use case; but heuristics would not cater to all needs

jan-ivar: my main comment it would be best to leave this to UA to experiment in this space before giving control to the app
… maybe with a special case for self-capture

elad: re UA experimenting - inside Google we already have differing needs from different apps that heuristics cannot handle

youenn: sourceChangeSupported would only be a hint? it would only affect browser UI to the UA discretion

elad: open to either
… in Chrome, we would abide by the hint

jan-ivar: would you also want to add an event to report the change of a source?

elad: not at the moment

youenn: this could be surfaced as configuration change

jan-ivar: I'm not supportive this, but looking at the defaults for a bit
… I think the constraint should default to allow source change (thus a preventSourceChange)

elad: works for me
… re not supportive - can you say more?

jan-ivar: not convinced we can't find good cross-apps heuristics?
… I'm hearing source change is harmful primarily on self-capture
… it shoudl also not apply to getViewportMedia

youenn: I think a hint would leave room for UAs to experiment, with a good default

jan-ivar: a hint avoidSourceChange limited to getDisplayMedia?

elad: sounds good to me

jan-ivar: sounds worth iterating on

dom: would be good to get more clarity on non-self-capture use cases; but the hint may be a good enough compromise
… usage of capture handle could be part of heuristics

elad: I ack that this is mostly interesting for self-capture

youenn: then this could be left to developers to pick getDisplayMedia vs getViewportmedia

elad: but getDisplayMedia is still a long way before being available

jan-ivar: but adding features to getDisplayMedia when we know it's unsafe, which removes incentives to getViewport media

elad: but we can't wait until getViewportMedia shows gaining adoption

jan-ivar: would be OK with a hint that we could deprecate
… not super excited about it, but won't block it either

elad: rough agreement!

WebRTC 🎞︎

Simulcast issues 🎞︎

[Slide 20]

Byron: ultimately, this cluster of issues boils down to: the Simulcast IETF spec means we have to allow a remote description to remove a rid at will
… the spec has a mix of language that seems to disallowing it and other dealing with it at least for initial answers
… we've got to decide how to handle this discrepancy
… we *have* to allow removal from a remote description; adding new simulcast encodings - we're not forced to allow this, and I don't see a reason to allow it
… unless there are concrete use cases to add encodings to the simulcast envelop after the initial negotiation, I don't think we should allow it
… does anybody has a concrete use case for it?

bernard: couldn't think of one

byron: the other somewhat harder wrinkle is SDP negotiation can reorder RID at will
… the meaning that the order has in the simulcast attribute is related to transmission priority which isn't really something webrtc-pc deals with
… the simulcast spec is somewhat handwavy on this, with a SHOULD we could ignore
… but with a reoffer with a different order of RIDs, we kind of have to re-use the same order in the answer even if it doesn't impact the [[SendEncodings]] slot
… as discussed in #2723

youenn: could we reject on a different order to simplify? or are there use cases for this?

byron: with e.g. an invalidmodificationerror?

youenn: yes

byron: that might be acceptable; I don't expect this would cause too much trouble practically
… this may be a spec violation
… mirroring the order without messing with [[SendEncodings]] is probably not too hard to specify

Bernard: let's refine the recommendations in the issues

Byron: particularly looking for input on #2723
… to open the path to a PR

Bernard: thanks! Simulcast raises a number of complex issues!

Next meeting 🎞︎

Bernard: scheduled on June 7; get in touch if you want to suggest agenda items

Minutes manually created (not a transcript), formatted by scribe.perl version repo-links-187 (Sat Jan 8 20:22:22 2022 UTC).