Meeting minutes
Recording: https://
Slideset: https://
Future meetings 🎞︎
Bernard: OK with moving up our next meeting to June 7th?
WebRTC 🎞︎
RTP Header Extension Encryption 🎞︎
RTP Header Extensions Encryption (cryptex) #106
Bernard: cryptex has completed IETF last call, with Sergio incorporating changes from comments
[slide 13] an API proposal from Sep 2020 meeting
… with RTP headers being encrypted as all or nothing per bundle
… with a policy on whether this encryption is negotiable or required
RTP Header Extensions Encryption (cryptex) #106
Bernard: please review the PR
… that adds this API
… a little complicated since it needs change patching the webrtc-pc steps
Jan-Ivar: any reason it's not a readonly getParameters
Bernard: it is
… it's set per transceiver (although it's really per bundle)
youenn: lgtm
… could we simplify by not doing it per transceiver?
bernard: how would you handle the situation where you receive an offer with different setting on different bundles?
… bundles don't get surfaced in the API
Byron: it could be a property on the transport, which is what represents the bundle
Bernard: please review the PR and bring your comments there
Jan-Ivar: what happens if this gets changed with setConfiguration?
Byron: really a bad idea :)
Bernard: should we prevent that from happening?
Byron: +1
… changing that is really awful, and so would writing tests for it
… let's keep it simple in absence of a use case for it
Issue #2735: webrtc-pc does not specify what level of support is required for RFC 7728 (RTP pause) 🎞︎
Bernard: this is about confusing about RFC 7728 (rtp stream pause) and the level of required support for it
… this isn't required
Byron: there is one place in the spec that refers to it in the API spec
… supporting the ~ flag requires support for 7728
… there seems to be support towards instead removing support for that flag
Bernard: not even sure why we would want to support it in the SDP
Bernard: I think it shouldn't be there; let's make this as ready for PR and figure out the right pull request to fix this
CaptureController 🎞︎
#190 Conditional Focus [screen-share]
Jan-Ivar: a proposal that helps addressing 3 issues, starting with https://
#57 .sendCaptureAction() misplaced [capture-handle]
#12 .getCaptureHandle() misplaced? [capture-handle]
Jan-Ivar: similar to Fetch's AbortController
… this can give more leeway on the acceptable time to give focus
Elad: the controller patterns looks nice
… would you be open to exposing a getter of the controller on the track?
… if an API returns several streams, would it return several controllers?
jan-ivar: on the first question, not exposing it on the track would be a feature
… adding a getter could be easily done by apps any way
elad: makes sense
jan-ivar: I haven't thought about multi-capture yet - happy to look into it; I would expect one controller per capture
elad: different type of capture (e.g. tab, window, screen) may require different type of controller
… e.g. focus on a screen doesn't make a lot of sense
jan-ivar: for "monitor", focus() would resolve immediately - essentially a no-op
elad: with subclassing, we could determine whether there is a focus() method - screen-subclass or audio-tracks wouldn't have it
jan-ivar: that's worth discussing - subclassing feels a bit excessive at this point, but to be discussed
youenn: separating functionality specific to a source from the track is a good idea
… adding it to track isn't a great model
… I'm less sure about the API surface, but we can iterate on it
… focus() is one thing; supported actions is a different thing
… they may require different objects
… e.g. actions may benefit from being transferable
… whereas focus handling doesn't really make sense
elad: how quickly do we think we can agree on an API shape?
… I wouldn't want this to delay other long-started discussions e.g. on conditional focus
Jan-Ivar: not big fan of subclassing; making the type of capture source detectable would be more useful
dom: maybe let's focus on using this pattern for conditional focus
elad: how do we deal with feature detection?
youenn: through the presence of the interface, as for other APIs
elad: not necessarily convinced, but +1 if it helps moving faster
dom: should this be discussed as part of screne-share repo then
jan-ivar: I'll create an issue in screen-share to help discuss the API shape before we move to a PR
WebRTC Encoded Transform 🎞︎
youenn: we agreed to add a generateKeyFrame API
… adding the frame timestamp would be a good addition, but this may result in having to have multiple timestamps when dealing with multiple RIDs
youenn: proposing instead to make the method apply to a single rid
aboba: lgtm
youenn: ok, so will finalize the PR
Extensions to Media Capture and Streams 🎞︎
PR #61: Add support for background blur and configuration change event 🎞︎
youenn: lots of background blur done with canvas and WebGL today
… OSes are now adding support for background blur
… the intel team has made measurements showing that OS background blurs gives a 2x power improvement over JS-based blur
… also, if the OS is already providing it, makes no sense to add another layer of blurring
youenn: proposal is to add a backgroundBlur constraint, a capability & a setting
… this would be a boolean constraint, as a starting point
… a double [0,1] range may be worth discussing, but I suggest starting with a boolean approach
… interested in feedback in the approach, and on the boolean vs double discussion?
jan-ivar: lgtm
riju: would shifting to double later be an option?
youenn: not sure about the migration path; it could be a new constraint which would work on a clear definition
… hearing support, so we can finalize and merge the PR then
youenn: the same PR #61 also includes a configuration change event
… since the OS setting can change outside of the UA control
dom: how broadly would this event apply? any constraints? a subset?
youenn: any track setting
jan-ivar: how would handle "exact" on a backgroundBlur applyConstraint?
youenn: if set at the OS level, it would reject
jan-ivar: ... but that only applies to "exact" constraint
youenn: right
jan-ivar: a UA could implement this and provides this per-track?
youenn: good question to discuss
… on iOS if you disable echoCancellation, it can't be disabled for a single track - so there are precedents
jan-ivar: apps can turn this on & off with applyConstraints - is that OK?
youenn: the browser could keep control on that within the constraints framework
florent: which capabilities would trigger configuration change?
… you mentioned OS level ones
… what if a device is open in several browsers and one would change e.g. the exposure?
youenn: we usually try to hide this - so the configuration change event will be at the discretion of the UA
… if two pages are capturing, the UA is in control
… but we need to be careful about broadcasting events across origins
jan-ivar: [on the chat] similar to devicechange we should add fuzzing
florent: +1 otherwise it could be used as a communication channel
youenn: florent, could you comment in that direction on the PR?
… hearing consensus in general on PR #61
jan-ivar: rough agreement at least :)
PR #59: Add powerEfficientPixelFormat constraint 🎞︎
youenn: submitted by henrik
… some cameras generation MJPEG video frames, which can have a negative impact on power consumption
… proposal is to add a new constraint to help
jan-ivar: sounds a good approach
dom: when would you not want this?
youenn: for specific resolution capture, or e.g . for a single frame capture
… the issue is that it's hard to have a good default with backwards compat
… there is leeway for the UA to pick an efficient camera
dom: but then shouldn't this be the UA doing the right thing rather than asking all developers to update their code?
jan-ivar: UAs might have different defaults on desktop vs mobile
… with exposing the info is getSettings, you can decide to make different compromise in terms of resolution vs power consumption
dom: getSettings I understand; device selection feels icky
youenn: it wouldn't be allowed as a required constraint in getUserMedia
jan-ivar: +1 to that - we don't want to exclusde people that only have MJPEG cameras
dom: hearing we should be doing this
Dynamic Source for Screensharing 🎞︎
Elad: apps may not expect the content of the source to be changing under their feet; e.g. in the context of a capture handle
elad: when the browser provides a feature that apps can't predict or control, this creates a bad experience for app developers (and ultimately end users)
youenn: I'm a bit surprised by the API surface; I'll need thinking some more, maybe the UA could do some heuristics
… not sure an app can determine whether it wants to support this at the time of getDisplayMedia
… have you discussed a more flexible API?
… or is it hyperfocused on purpose?
elad: I'm up for giving more flexibility to apps
… re heuristics, they're not going to serve well all apps equally
youenn: what would be the default?
elad: I would argue that backwards compat should default to the existing behavior (not dynamic sources)
… but open to discuss this
jan-ivar: a bit concerned that it would let the application limit the user choice
… I like the feature in Chrome, dynamic switching is a good feature
… also questions when source changes whether focus should be set
… not sure if we shoud allow any app decide whether or not to allow the user to switch
… how much of the use case is tied to the selfCapture constraint
… re capture handle, capturing a different tab is not much different from navigating from a captured tab
elad: re allowing limiting user selection - right now, there is no choice offered to the user, this would add a new choice
… re focus, with the current Chrome implementation of the UX, you can only press the button when the tab is focused, so it's not an issue for us at the moment; may need discussions if that changes
… re self-capture, you're right that's the clearest use case; but heuristics would not cater to all needs
jan-ivar: my main comment it would be best to leave this to UA to experiment in this space before giving control to the app
… maybe with a special case for self-capture
elad: re UA experimenting - inside Google we already have differing needs from different apps that heuristics cannot handle
youenn: sourceChangeSupported would only be a hint? it would only affect browser UI to the UA discretion
elad: open to either
… in Chrome, we would abide by the hint
jan-ivar: would you also want to add an event to report the change of a source?
elad: not at the moment
youenn: this could be surfaced as configuration change
jan-ivar: I'm not supportive this, but looking at the defaults for a bit
… I think the constraint should default to allow source change (thus a preventSourceChange)
elad: works for me
… re not supportive - can you say more?
jan-ivar: not convinced we can't find good cross-apps heuristics?
… I'm hearing source change is harmful primarily on self-capture
… it shoudl also not apply to getViewportMedia
youenn: I think a hint would leave room for UAs to experiment, with a good default
jan-ivar: a hint avoidSourceChange limited to getDisplayMedia?
elad: sounds good to me
jan-ivar: sounds worth iterating on
dom: would be good to get more clarity on non-self-capture use cases; but the hint may be a good enough compromise
… usage of capture handle could be part of heuristics
elad: I ack that this is mostly interesting for self-capture
youenn: then this could be left to developers to pick getDisplayMedia vs getViewportmedia
elad: but getDisplayMedia is still a long way before being available
jan-ivar: but adding features to getDisplayMedia when we know it's unsafe, which removes incentives to getViewport media
elad: but we can't wait until getViewportMedia shows gaining adoption
jan-ivar: would be OK with a hint that we could deprecate
… not super excited about it, but won't block it either
elad: rough agreement!
WebRTC 🎞︎
Simulcast issues 🎞︎
Byron: ultimately, this cluster of issues boils down to: the Simulcast IETF spec means we have to allow a remote description to remove a rid at will
… the spec has a mix of language that seems to disallowing it and other dealing with it at least for initial answers
… we've got to decide how to handle this discrepancy
… we *have* to allow removal from a remote description; adding new simulcast encodings - we're not forced to allow this, and I don't see a reason to allow it
… unless there are concrete use cases to add encodings to the simulcast envelop after the initial negotiation, I don't think we should allow it
… does anybody has a concrete use case for it?
bernard: couldn't think of one
byron: the other somewhat harder wrinkle is SDP negotiation can reorder RID at will
… the meaning that the order has in the simulcast attribute is related to transmission priority which isn't really something webrtc-pc deals with
… the simulcast spec is somewhat handwavy on this, with a SHOULD we could ignore
… but with a reoffer with a different order of RIDs, we kind of have to re-use the same order in the answer even if it doesn't impact the [[SendEncodings]] slot
… as discussed in #2723
youenn: could we reject on a different order to simplify? or are there use cases for this?
byron: with e.g. an invalidmodificationerror?
youenn: yes
byron: that might be acceptable; I don't expect this would cause too much trouble practically
… this may be a spec violation
… mirroring the order without messing with [[SendEncodings]] is probably not too hard to specify
Bernard: let's refine the recommendations in the issues
Byron: particularly looking for input on #2723
… to open the path to a PR
Bernard: thanks! Simulcast raises a number of complex issues!
Next meeting 🎞︎
Bernard: scheduled on June 7; get in touch if you want to suggest agenda items