W3C

– DRAFT –
WebRTC Interim, May 21st

21 May 2024

Attendees

Present
Anssi Kostiainen, Bernard Aboba, Carine Bournez, Elad Alon, Florent Castelli, Fredrik Solenberg, Guido Urdaneta, Harald Alvestrand, Jan-Ivar Bruaroey, Michiel De Backker, Paul Adenot, Peter Thatcher, Sameer Vijaykar, Sun Shin, Tony Herre, Tove Peterson, Youenn Fablet
Regrets
Dom
Chair
Bernard, HTA, Jan-Ivar
Scribe
caribou

Meeting minutes

Recording: https://www.youtube.com/watch?v=SsruLUC6uew

Slideset: https://lists.w3.org/Archives/Public/www-archive/2024May/att-0003/WEBRTCWG-2024-05-21.pdf

Captured Surface Control (Elad Alon)

Elad: API shape proposal, slide 13

[Slide 13]

[Slide 14]

Elad: Any browser support zoom levels that are reasonable

[Slide 18]

Peter Thatcher: I'd propbably think of changing the slides, not just scrolling/zooming

Elad: it's right that other things could be possible

Jan-Ivar: What you showed is useful. UC are appealing
… remotely controlling user content might be troublesome

Elad: it's difficult to put the controls at the browser level

Elad: most convincing case is with multiple scrollbars

Peter: the JS application controls the scrolling from another tab?

Elad: true

Peter: I don't know what the scrolling event do in the page
… not sure the users understand that

Elad: you don't do screen sharing with any app
… the context is more constrained
… there's a mention saying "this is shared and can be scrolled and zoomed"
… and that offers revokation

Youenn: more intergration would avoid prompting
… with media elements, there are elements that can be showed to users
… some web devs used them, some put eveything to off and reimplement
… a hybrid approach between prompting and media control would be good

Elad: I'd like to enable apps to put the zooming control wherever they want

Youenn: CSS applied to the text track can be user agent specific
… this idea can be reused there

Elad: I understand the benefits but it
… is limiting the UX
… Scrolling is more difficult, so maybe we should split discussion with zooming

Youenn: you can zoom screens, windows, etc.
… we'd need to be sure it is applicable

Elad: our next step is to work on shape for zooming windows

Bernard: Region capture restricts sharing to a portion of screen
… how does this combine with zooming/scrolling?

Elad: the application still has access to all the pixels but removes the cropped zone

Bernard: if I call cropTo , will the scrolling stop at the boundaries?

Elad: we can discuss interaction of those APIs

Jan-Ivar: in the UI exposing the controls
… [that's visible in slide 12]
… I'd love to pursue an approach where the application is choosing where to show the indicator

Elad: you're supposed to trust the application

Guido: @@@

Elad: are there any measures as security gap?

Youenn: on event click

Elad: after the prompt, we don't require user activation

Jan-Ivar: it'd be good to understand the remote user UC

Elad: I trust the videoconferencing app to be safe

Youenn: you already say that some restrictions on the API
… would not go well with remote control
… so the model seems to be for remote control UC
… maybe we do NOT want remote control
… so not loosen security

Elad: It's not clear why we want to block remote control

Jan-Ivar: any kind of remote control has security concerns

Elad: it's not proven that this API would increase the ability to trick people

Jan-Ivar: most scammed people are told to download remote control apps

Elad: this API does not has a click API

Bernard: I agree with Florent that we should have no keyboard events either
… very well-defined limits is essential

Elad: I'd like to discuss whether it's the browser or the app that offers
… next steps?
… discussion on whether to outscope remote control

Youenn: where is it?

Elad: screen capture CG so far

RESOLUTION: continue discussion in the screen capture CG repo

Issue 225: Add captureTimestamp senderCaptureTimeOffset to the encoded frame metadata

[Slide 22]

[Florent Castelli presenting]

Florent: this is for calculation of the delay

Bernard: it looks like a good idea
… it would be useful to construct encoded frames

Youenn: the definition of locally captured frame needs to be precise

Bernard: next step is to review. any objection to this?

RESOLUTION: next step on issue 225 is to review the PR

Issue 226: Expose RTCEncodedAudioFrame interface in Worklets

[Slide 23]

Youenn: with an audio worklet, you'd want to share the buffer
… a variant of shared buffer would be a readable stream
… you push the read buffer from the worklet

Florent: it would be approach (1), but it might be a problem for performance
… so preference to run everything in the worklet

Youenn: it's worth getting numbers

Florent: yes, we want to get more data to make sure it's the right thing

Paul Adenot: as an Audio WG person, I'd say that the 2nd proposal cna't be done
… because it requires GC
… 1st is likely to work with higher performance
… we can do BYOB

Paul: can someone send a link to a BYOB approach, for the audio WG?
… it seems to have the right properties
… if you need something from the Audio WG, please open an issue there

Issue 1003: repo name nit: it'd be nice if this were simply w3c/mediacapture

Jan-Ivar: this is an chair/editorial? issue
… can this repo name be changed to be aligned ?

Harald: when we split WebRTC and mediacapture streams, we thought mediacapture would be the main spec, hence the "main" here
… renaming is possible but lots of links might break

Elad: GH gives you redirect

Anssi: speaking of experience, make sure to redirect all

RESOLUTION: Mediacapture-streams will be renamed to Mediacapture

Issue 966: Should devicechange fire when the device info changes? (Jan-Ivar)

[Slide 28]

Youenn: I don't think safari violates this
… Safari is exposing fake devices
… when the user allows, the real list is exposed
… it's allowed by the spec

Jan-Ivar: the stored device list is not changed?

Youenn: It is changed
… it's not sending the real list before it's allowed

Jan-Ivar: I'll amend this slide.

[Slide 29]

Guido: is the device list event enough to see the change?

Jan-Ivar: the advice is to compare the lists
… but you can't distinguish when a user really added a device
… or if it's a change in exposure

Guido: so you'd fire 2 events?

Jan-Ivar: yes

Youenn: suggest to just use a flag on existing event
… so just devicechange is sufficient

Jan-Ivar: how do you know which specific device is changed?

Youenn: maybe more than a boolean flag needed
… I'd like to extend the existing event

Guido: same feedback, it's backwards compatible
… not sure if a boolean is enough, we'll give a bit more thinking

Local Peer-to-Peer API (Anssi Kostiainen, Michiel De Backker)

[Slide 32]

Anssi: this is developed in WICG
… Driving motivation is trust on local network
… Martin Thompson also explored https on local network

[Slide 33]

[Slide 34]

[Slide 35]

Michiel: it's quite similar to WebRTC architecture

[Slide 36]

Michiel: Feedback, Questions?

Bernard: we have a few minutes

Youenn: my initial reaction is that local discovery is usually behind OS security protection

Michiel: we use permissions
… for each origin

Peter: I really like the use cases
… also the idea that we can use existing APIs
… looking foward to participate

Florent: I hope that we won't duplicate APIs with just little differences
… Can it be used to commumincate between 2 tabs from a local browser?

@@@

Harald: why replacing all components?
… transporting rtp packets is well known
… rtp over QUIC
… separate components can avoid replacing

Michiel: we want to be able to introduce the least amount of new things

Harald: if you have a key exchange mechanism it would be a smaller change
… in webRTC the media part is relying on dtls

Jan-Ivar: I don't think data channel is going to help
… it's derived from web sockets
… similarity is superficial
… Web Transport is currently Client-Server

Bernard: let's do something at TPAC

Anssi: We welcome new contributors
… let's keep exploring. thank you for the time

RtpTransport (Peter Thatcher)

[Slide 56]

Explainer

More on congestion control etc.

[Slide 70]

Peter: feedback requested

Are all of your (relevant) use cases covered?

If not, file an issue at https://github.com/w3c/webrtc-rtptransport/issues

Would you like to comment on the question of Workers?

Go to w3c/webrtc-rtptransport#33

Would you like to comment on the API shape while maintaining perf

Go to w3c/webrtc-rtptransport#20

Have any other thoughts/ideas?

File an issue at https://github.com/w3c/webrtc-rtptransport/issues

Bernard: [referring to slide 56] Your AC seems to cover all 4 webRTC extended UCs

Peter: the 1st explainer covers a lot

Youenn: I like the focus on custom packetization
… we'll probably need a definition of a "packet source"
… e.g for the new encoded video frame
… About worker/not worker, it depends on whether you transfer dynamically

Bernard: UC for dynamic transfer?

Youenn: not really. only very complex
… a fixed place is simpler and covers most cases

Harald: a frame level API is more convenient to work with

Jan-Ivar: there are a lot of security issues
… security models of browsers ...
… I hope we can integrate existing tech more
… API-wise, we have similar low-level APIs in webRTC

Youenn: I had similar concerns on encoded transform

Bernard: I think the plan is to refine the UCs
… the Game Streaming UC may not belong here

Bernard: this can be a TPAC topic

Bernard: ADJOURNED

Slideset: https://lists.w3.org/Archives/Public/www-archive/2024May/att-0003/WEBRTCWG-2024-05-21.pdf

Summary of resolutions

  1. continue discussion in the screen capture CG repo
  2. next step on issue 225 is to review the PR
  3. Mediacapture-streams will be renamed to Mediacapture
Minutes manually created (not a transcript), formatted by scribe.perl version 222 (Sat Jul 22 21:57:07 2023 UTC).