WebRTC February 2024 meeting

20 February 2024


Bernard, Carine, Dom, Elad, Fippo, Florent, Guido, Harald, Jan-Ivar, PatrickRockhill, Peter, SunShin, TimP, Youenn
Bernard, HTA, Jan-Ivar

Meeting minutes

Recording: https://youtu.be/_LGtX7deM7o

Slideset: https://lists.w3.org/Archives/Public/www-archive/2024Feb/att-0008/WEBRTCWG-2024-02-20.pdf


Issues with receive-only codecs aboba/hevc-webrtc#22 🎞︎

[Slide 11]

Bernard: not a WebRTC issue - it came up in IETF AVTCORE wrt receive-only codecs

Repository: w3c/webrtc-pc

Modify the codec description model to ease describing changes #2925 #2935 🎞︎

[Slide 12]

[Slide 13]

[Slide 14]

Harald: is this a reasonable direction to go in?

Bernard: it is; the more I read, the more current situation doesn't make sense, with too much unspecified, so this is a good step forward

Jan-Ivar: overall it makes sense to add more details to it and it will help having a more neutral baseline
… we need to keep track of fingerprinting concerns, but we should be aligning with Media Capabilities in any case
… this looks like an improvement to me

Bernard: Media Capabilities doesn't have any way to look at directionality except encoder/decoder - not sure if it's a good match

Harald: you have to make two calls to media capabilities to figure what you can send/received

bernard: so you're confident that encoder/decoder matches send/receive? we should test it too

Harald: hearing no push back, we'll bring this as candidate to merge at the editors meeting on Thursday

RESOLUTION: Bring #2935 for merging to editors meeting

Should media capabilities influence what is exposed in what is exposed in WebRTC offers and answers #2929 🎞︎

[Slide 15]

Youenn: more and more codecs get exposed on the Web and in WebRTC
… for playback, media capabilities allow to control how much information gets exposed to the web site
… for WebRTC, everything gets exposed via getCapabilities or via SDP
… this is a fingerprinting issue, but also has an impact on the number of allocated payload types

Harald: not all codecs get exposed, but they can be discovered through setLocalDescription

Youenn: getCapabilities was initially designed to expose everything supported, but it would be best to deprecate it
… can we move to use media capabilities as a replacement?

[Slide 16]

Youenn: please chime in in #2929 too

Bernard: this makes sense for the simpler codecs à la VP8, VP9, AV1, but e.g. HEVC raises weird situations
… say I query a common level that can be used for encoding/decoding, how would this impact the whole offer/answer negotiation?

Youenn: the idea would be that media capabilities would give you WebRTC-specific data that could be plugged into the WebRTC API

Bernard: but for these send-only or receive-only situations across levels...

Youenn: SDP can't express all these situations
… this wouldn't be an improvement, but it doesn't make things worse
… and at least, the web app would know what's not available e.g. on the decoding side

TimP: I like this - it feels overdue
… the UA-to-UA disadvantage doesn't feel too serious
… the only issue I have is how precise the query would have to be e.g. on which submode of which profile of a codec
… esp since UA are known to lie on some of these questions

Bernard: indeed, e.g. with H264 and 265

Youenn: this profile complexity would be in scope indeed

Harald: it's probably good to link webrtc and media capabilities codec more closely
… Youenn has a PR to have media capabilities return a WebRTC shape for doecs when queried
… to ensure it is inspectable and usable in WebRTC land
… it's good to have the default set of codecs be implementation defined
… if we have a codec that is universally supported by a given platform, it doesn't expose much fingerprinting surface to expose it

Youenn: I'll revive the Media Capabilities proposal

Jan-Ivar: I'm also supportive; reducing fingerprinting surface feels good, or at least reducing the list to a default list

Youenn: I'll work on an API proposal to tie Media Capabilities and WebRTC - not sure yet if it's at the transceiver level, please bring input on the issue

Harald: the codec model description will help as well for this

RESOLUTION: Proceed with proposal based on data-minimization appraoch

RESOLUTION: Youenn to revive to Media Capabilities proposal media-capabilities#186

Existing setCodecPreferences NOTE is wrong and should be deleted #2933 🎞︎

[Slide 17]

Fippo: if no objection, I'll bring a PR and a WPT test

Bernard: I agree it's extraneous

RESOLUTION: Fippo to submit a PR to remove note and a matching WPT test

setCodecPreferences to deal with both send and recv codecs #2939 🎞︎

[Slide 18]

[Slide 19]

Bernard: I'm not sure this is entirely right given that it's based on a JSEP paragraph that looks like it may be wrong
… sendrecv, sendonly, recvonly are 3 separate sets that setCodecPreferences gives order to
… when changing direction, you're shifting to a different set
… it's not clear that the preferences can survive such a shift

Fippo: JSEP is confusing in that area
… it only talks about sendrecv, not the two other sets you're alluding to
… it may be challenging to change this in terms of web compatibility though

Jan-Ivar: regardless of what JSEP says, it would be desirable if we kept sCP and direction change orthogonal
… this could be addressed by having a super list that is filtered; if the filter ends up with an empty list, I'm a bit wary about throwing, although this does feel like a mistake

Harald: changing direction after you have configured the codec influences the set of codecs available
… if we require at least one sendrecv codec, we are safe, and adding recvonly codecs is also safe
… this is a painful aspect of SDP - we should just admit failure on the sendonly case and figure out the least painful approach to deal with - maybe the last option

Fippo: if we have nothing in common, the regular approach would be to reject the m-line, hence my preference for the 3rd option
… should we wait for a JSEP change in this space?

Bernard: I think we should come up with a proposal for JSEP

Fippo: OK, will bring this back at the next meeting after more off-line discussions

Screen Capture 🎞︎

[Slide 22]

Bernard: the WPT results for Screen Capture shows little adoption for capture controller

Jan-Ivar: lots of green (getDisplayMedia works in all browsers), the red parts are for the new capture controller API
… some red on iframe delegation, but the big difference is capture controller
… it's not controversial, but hasn't been implemented outside of Chrome yet
… it's not on FF's short term roadmap

Youenn: likewise for Safari
… we discussed a similar issue in Media WG - moving to CR would still be beneficial

Jan-Ivar: +1

Issue triage and milestones 🎞︎

[Slide 24]

Jan-Ivar: please chime in if you feel the assignment of issues to milestones need adjustment

Should we have a screenshare extension spec? #297 🎞︎

[Slide 25]

Youenn: I think it makes sense; in the Media WG, it was advised that an extension spec is more complex and needs more work for editors, but from the point of view for web developers, it clarifies what is mature and what isn't
… if it's not useful for developers, then maybe a forever CR would be a better model

TimP: do we think realistically it will help? are the 12 enhancements really blocking? are the 19 issues solvable in a reasonable timeframe?

Jan-Ivar: I think it will help because the enhancement are real additional features

Elad: +1 if it helps with making progress on issues

TimP: but do we thinkg the 19 CR issues are solvable in a sensible timeframe?

Jan-Ivar: yes

Elad: there may also be issues that shouldn't be considered as CR blocking

Jan-Ivar: getDisplayMedia mostly works, and separating what's additional value would help

Bernard: +1 that they're addressable, and they would allow to get e.g. more focus on privacy issues

Jan-Ivar: hearing mostly agreement this would be the right path forward

Elad: when do we want to lock the list of issues as CR-blocking?

Jan-Ivar: I think we can start moving issues over to the new repo right away, and move them back if needed
… and then we have to do the work to close the 19 issues

RESOLUTION: Create extensions repo for screen capture and move issues related to new features there

Distinguish cancellations from absent OS permissions #281 🎞︎

[Slide 26]

Elad: this is a problem worth solving; this sounds like a possible solution, wonder if we could improve it
… i.e. the spec isn't explicit on this, would be useful to make it so
… also not sure NotFoundError isn't the most explicit expression
… e.g. we could define a new error with a more specific type

Jan-Ivar: in general, there is generally pushback against defining new custom errors
… and prefer re-using existing DOM errors even if they're not a perfect fit
… I'm not too concerned about this imperfect fit

Youenn: NotFoundError feels like a reasonable minimal API already in use; in terms of ergonomics, I think it's ok as long as the spec is very explicit about this interpretation of NotFoundError

Jan-Ivar: +1 to making this explicit in the spec

Elad: can we make it that NotFoundError be restricted to this? e.g. in a situation where constraints would limit shareable surfaces?

Jan-Ivar: it sounds like NotFoundError would still be a reasonable fit for that situation, but that feels like an edge case

Elad: fair, we can leave that question aside
… could we still allow UA-dependent subclassing?

Dom: that would lead to non-interoperable behavior

Youenn: re OS permissions, I'm not sure that's a well-defined concept for the platform
… we should check what's being used e.g. in HTML or Permissions

Jan-Ivar: maybe this is a clarification to bring to mediacapture-screenshare on "no sources of type T are available" with a parenthesis that describes these examples

Jan-Ivar: would we want to apply to this mic/cameras? knowing that they could be distinguished from no hardware with enumerateDevices()

Guido: for OS permissions in camera/mic, we use notallowederror with a different message

Elad: enumerateDevices() isn't 100% robust to make the distinction given that users can plug/unplug devices

RESOLUTION: Move forward with a PR to clarify that NotFoundError would apply for OS permissions in screen-capture

Jan-Ivar: I'll file an issue for a follow up discussion on mediacapture-main

Elad: aligning the two would be best (although not fully required)

MediaStream Recording 🎞︎

[Slide 29]

Bernard: WPT shows tests with limited support

Youenn: Safari doesn't support webm recording; not sure about mp4

dom: I don't think there are codecs requirements in recording - if so, codec specific tests should be marked as optional

[Slide 30]

Jan-Ivar: overall numbers are looking pretty good

Youenn: not sure why there are codec/format-specific tests e.g. for stop()

[Slide 31]

Bernard: I support this; WebCodecs is indeed the way forward for many of the requested enhancements for recorder

[no objection raised to proceeding with that plan]

Issue #202 Deprecate isTypeSupported 🎞︎

[Slide 32]

Bernard: this could simply quote media capabilities spec

Jan-Ivar: we could return a fixed list as Youenn suggested earlier in the webrtc case
… this would solve the privacy issue neatly here

Bernard: wfm

Youenn: +1

Dom: it needs to stay in the spec for web compat, but should be described as returning fixed answers and its usage discouraged

RESOLUTION: Make isTypeSupported return a fixed list of answers and mark it as deprecated

Media Capture specs 🎞︎

Issue triage and milestones of Media Capture and Streams 🎞︎

[Slide 35]

Jan-Ivar: we would like to see more activity in this spec to accelerate progress towards Rec

captureStream on OffscreenCanvas on 🎞︎

[Slide 37]

Youenn: no objection - but feels like low priority, not sure there is much web developer demand for this

Harald: offscreencavas has usages, not all of them in workers
… this isn't related to mediacapture-main though?
… the only relation is the availability on MediaStream on workers

Jan-Ivar: yes, that's the next issue I wanted to discuss, since answering yes to this would give an answer to this

Jan-Ivar: not hearing objection, nor much interest either

RESOLUTION: Supporting captureStream on OffscreenCanvas is reasonable but low priority

Expose MediaStream in Workers 🎞︎

[Slide 38]

Jan-Ivar: not hearing objection on this either

Dom: (but not much excitement either)

RESOLUTION: Exposing MediaStream in workers is reasonable but low priority

How should MediaStreamTrack interact with BFCache? 🎞︎

[Slide 39]

Youenn: +1 to this proposal; not just because that's Safari current behavior, but also because it will help with getting web sites handle device failure situations better
… would like us to be more proactive on pushing outreach for this

Harald: I've had a lot of discussions on BF-cache in the context of peerconnection where we're trying to make WebRTC more BF-cache friendly
… This sounds nice, but I think I'll want to think this through some more

Youenn: this is indeed also worth discussing for WebRTC-PC

Harald: let's keep discussing this in the issue, want to hear from Guido

Guido: +1

Youenn: I'll file an issue in webrtc-pc on BF-cache friendliness

Add guidance for defining a new source of MediaStreamTrack 🎞︎

[Slide 40]

Jan-Ivar: this PR adds clarifications to what muted and ended for other sources of tracks

[Slide 41]

Guido: +1 on the generic guidance; the mic/camera language needs more discussion

Youenn: proposed language sounds good to me; we should review our source-defining specs to make sure they're consistent with that guidance

Jan-Ivar: yes, let's file these issues before landing that PR
… will get that done before the next Editros meeting

RESOLUTION: Bring #988 to editors meeting for merging after having filed issues in source-defining specs

Summary of resolutions

  1. Bring #2935 for merging to editors meeting
  2. Proceed with proposal based on data-minimization appraoch
  3. Youenn to revive to Media Capabilities proposal media-capabilities#186
  4. Fippo to submit a PR to remove note and a matching WPT test
  5. Create extensions repo for screen capture and move issues related to new features there
  6. Move forward with a PR to clarify that NotFoundError would apply for OS permissions in screen-capture
  7. Make isTypeSupported return a fixed list of answers and mark it as deprecated
  8. Supporting captureStream on OffscreenCanvas is reasonable but low priority
  9. Exposing MediaStream in workers is reasonable but low priority
  10. Bring #988 to editors meeting for merging after having filed issues in source-defining specs
Minutes manually created (not a transcript), formatted by scribe.perl version 222 (Sat Jul 22 21:57:07 2023 UTC).