WebRTC TPAC F2F Day 1 -- 19 Sep 2019

<dom> Meeting slides

<dom> ScribeNick: youenn

WebRTC F2F Thursday morning

Agenda review

State of the WebRTC WG

Slight difference between charter ending March 2020 and list of deliverables.

Probably not able to deliver everything

Web developers want interop, it should just work

media capture main spec: 26 issues, 21 of them sticking around, probably harder to fix.

community sense is that it works.

Going to PR requires closing bugs and validating testing.

screen capture spec

developer use it and sense is that it works.

We should get it to CR.

13 open bugs.

dom: need full horizontal review

Goal is to fix all CR blocking issues by ending of F2F

Need also to fill out TAG review.

Need an explainer for the TAG review

webrtc-pc!

32 open issues, still new ones coming in at a good pace

Bugs are smaller and smaller

<scribe> New PR target at the end of F2F

WebRTC-identity

Not much progress there

WebRTC 1.0 Testing

Specs have links to tests, should allow to compute spec coverage.

A lot of progress in tests passing in all 4 browsers.

DrAlex presenting the WPT and KITE dashboards

Simulcast testing big progress with IETF 104 and 105 hackathons.

Next steps to PR for WebRTC-pc

Chrome has mid and rid header extensions, Firefox supports rid header extension.

Florent implemented the simulcast playground for unified plan.

Could be used as a basis for WPT.

<scribe> ACTION: Florent to investigate upstreaming his page as WPT test

<trackbot> Created ACTION-124 - Investigate upstreaming his page as wpt test [on Florent Castelli - due 2019-09-26].

<dom> Confluence-based WebRTC Implementation tracker

Plan: do a final CR containing all new changes, then no substantive changes. Then validate testing is sufficient to go in PR.

Goal is to go to PR by end of the year.

Issue 1764

Bernard summarizing the issue

Discussing Nils proposal.

Harald: are all codecs ok with sending, then not sending and resending audio packets?

Opus: you need to process a few packets before outputting sound

Varun: what about doing like in mute case?

Bernard: send silence is not very clear, might change according the negotiated parameters.

RESOLUTION: Accept Nils proposal for Issue 1764

Issue 2121

RESOLUTION: Proposal C: define retrieval of constraint properties for getSettings but make them not modifiable

<jib> for width, height and frameRate, and aspectRatio

Issue 2230

<dom> trackbot, bye

<dom> Youenn: the host field in the ice error event exposes private ip addresses

<dom> ... in theory it could expose private information; in practice, it's exposing addresses that in general would have already been exposed to the page

<dom> ... the spec opens up the possibility to use 0.0.0.0:0 if filtering is needed

<dom> ... there is an odd case when you ask only for relay candidates

<dom> ... we could maybe filter the host address in that case

<dom> ... returning 0.0.0.0:0 instead of null is a bit strange

<dom> ... and why is port and address in a single field?

<dom> jan-ivar: maybe either than null, an empty string?

<dom> harald: the advantage of 0.0.0.0:0 is that it is parseable by an IP address parser

<dom> ... e.g. if you log the errors

<dom> dom: since it has been very recently deployed, this is purely a cosmetic design decision at this point

<dom> jan-ivar: empty string sounds good to me

<dom> henrik: we should be consistent with what we have in RTCIceCandidate

RESOLUTION: empty string for null value, split port separately

<dom> (make port nullable, find name for host)

<dom> ISSUE 2257

Same certificate for workers so postMessage is potentially useful.

Bernard: for forking, same certificate for different endpoints, potentially in a worker thread.

RESOLUTION: Keep the existing ability and put a note about the security issue.

postMessaging a certificate might be useful in a data channel context so it might be a useful feature.

Screen capture

Jan-Ivar is listing the list of improvements done in the past year.

Issue 60: Define Tab Capture

RESOLUTION: Use 'browsing context' as tab capture

Need horizontal review, fill out privacy and security questionnaire. Need explainer.

Discussion about tab capture. Mozilla might want to capture the active tab (without the other tabs like a window would leak).

Potential clarification: make it clear that tab capture might change the the tab dynamically.

Media capture and streams

Issue 554

Harald: lack of spec actually makes it harder to implement. Definitely interested and would like to push implementation when there is a spec.

Jan-Ivar: Would be good to have for our own testing.

<dom> Issue 565

RESOLUTION: allow browsers not to fire devicechange event if the list of device is actually the same

<dom> make this mandatory behavior unless this can be privacy-neutral

Issue 584

Safari has potential issues with downscale restriction in multiple apps accessing to camera.

RESOLUTION: Remove upscale from the proposed sentence and merge

Issue 608

Media Recording

ISSUE 139

Proposal is to always re-encode when mimeType is set and not reencode otherwise.

RESOLUTION: Consensus for Proposal B

Issue 167

RESOLUTION: Start with proposal B, Jan-Ivar to create a PR, promise based.

Need to discuss multiple track change cases.

WebRTC Stats

<dom> henrik: [implementation status] - the gaps in implementation reflects more structural changes in where the stats are exposed rather than the lack of available metrics

<dom> ScribeNick: steveanton

youenn: wpt should be enough for simple counters
... WebKit has some basic wpt validation that can be upstreamed

hbos: chrome does not have detailed tests at the integration level

<dom> ACTION: Youenn to upload simple validation tests from webkit to WPT

vr000m: explore simple tests in wpt then go forward from there

bernard: new model better supports SVC

issues 361/452

general agreement to proceed

issues 478/396

youenn: can already get this information from the api

jib: opposed for stats reporting things that the app already knows

hbos: need mid to correlate stats objects with the m= section (proposal 2)

jib: adding transceiver stats requires adding transceiver id

bernard: snapshotting state would help correlate stats with changes in signaling

jib: worried about bloat in stats objects

vr000m: two issues (1) do we need a transceiver dictionary (2) once we have that do we need additional data e.g. direction

jib: could also just put a mid on sender/receiver

vr000m: stats are used largely for debugging

agreement that mid should be mti

jib: opposed to direction/currentDirection more than opposed to transceiver stats

general agreement to add transceiver stats to with mid/sender id/receiver id only

<vr000m> also codecIds?

issue-480/issue-472

bernard and jib sold on proposal 1

vr000m: like events since we then we don't have to shim peerconnection

youenn: prefer that stats expose replace track info directly rather than through events

jib: see that shimming replacetrack is not ideal, but does work

vr000m: not providing something like onstatsended leaves a gap with replacetrack that has been known for 2 years

youenn: onreplacetrack should be a follow up and not in 1.0

RESOLUTION: remove onstatsended and continue discussion about an event to notify when significant stats change

issue-479

RESOLUTION: consensus to remove track stats (moving it to obsolete section)

Content-Hints

issue-2248

issue-28

issue-28 & issue-30

hta: more sympathetic to normative action than before

hbos: agree that we should only ship APIs that are well defined

jib: trying to be agnostic about what to do next, just want to provide feedback

bernard: where to specify normative hints for content hints?

jib/hta/bernard: content hints should be a follow up spec that covers the integrations with other specs

youenn: is a high level api that influences control knobs the right design?
... if hints are crucial for an application then we should specify the behavior

jib: hints can be useful since they are simpler than turning all the knobs

disposing of content-hint

bernard: some advanced codecs have content capture tools (hevc, av1)
... useful to expose this for e.g. screensharing

RESOLUTION: advance but not at high priority

DSCP API

RESOLUTION: proposal 1

(Move Priority to DSCP document, harmonize, keep experimenting)

WebRTC-ICE

youenn: not as opposed to adopt as last year, but not sure if it's the right time (don't want to delay from finishing 1.0)

dom: shouldn't be a high cost to the working group

generally no opposition to adopting the spec

bernard: do the p2p mesh folks want datachannels in workers?

dom: does this expose new security & privacy risks?

peter: need to think through the risks more

bernard: forking has been thought through, but not in the context of p2p mesh
... who is asking for ICE forking? just dht people?

hbos: why is ice forking hard?

bernard: each transport has different candidate pair statuses, API changes (dtls transport) to facilitate forking, more ICE optimizations that are more important for forking
... might not be that complex if you started from scratch and knew what you wanted to do

peter: two code bases which would need to implement forking

no current plans for either to do it

hta: not hearing much enthusiasm in the room

jib: more use cases could motivate implementors

bernard: p2p mesh is an issue in the use cases repo

Features at risk

<dom> ScribeNick: dom

JIB: Overview of feature at risks: there are about 12 features marked at risk
... I'll present some of our options to manage features at risks
... if no implementation and no dev interest, we remove the feature
... if no implementation but dev interest, we can leave the spec if we expect imminent implementation
... if not, we can move to an extension
... We conducted an analysis of what was implemented where to identify which features should be at risk
... focusing on things with fewer than 2 implementations and no clear implementation commitment
... for instance, rollback has only 1 implementation, but there is one ongoing
... [reviewing the list to check for possible mistakes]
... VoiceActivityDetection is at risk - only one (buggy) implementation

Bernard: does it negotiate CN?

Henrik: checking it now

JIB: OauthCredential is not implemented anywhere

Henrik: re VoiceActivityDetection, what Chrome ships can be done via setParameters

JIB: pranswer - 2 impl

Youenn: not sure our implementation should really count; this may need to be marked at risk

JIB: we're interested in implementing, but it hasn't been a priority
... QoS priority is at risk
... icecandidateerror?

Henrik: it's shipped in Chrome and old-Edge

Youenn: will need to be tested

JIB: RTCError isn't shipped yet anywhere

Henrik: we have the basic plumbing, but finalizing it is more work than you would expect

Dom: how confident are we about this landing any time? any risk of later interop?

Youenn: pages could break if they've started relying on existing error names
... Should we instead specify what currently ships (if interoperable)

Bernard: we designed RTCError to provide more details
... Errors in operational services occur at a very high rate

Dom: let's plan on discussing this tomorrow

JIB: simulcast-aware stats?

Henrik: spec is ready

JIB: commitment from Edge and Firefox to implement
... Are they MTI?

Henrik: adopting them means changing the behavior in a backwards incompatible way

JIB: statsended we decided to remove
... setStreams?

Orphis: in Chrome

Youenn: planned for us

JIB: us too, by the end of the year
... RTCPeerConnectionIceEvent.url is only in Safari - any other implementation commitment?

Bernard: I think we would want to do this for Edge/Chrome

JIB: planned for us, but probably not by EOY
... Identity - in Stockholm, we reached a compromise to split identity into a separate spec
... there are still a few normative ref from -pc to -identity
... but still only one implementation
... are we ok with that situation?

Youenn: I think it should be marked at risk

JIB: getDefaultIceServices - not implemented should be marked at risk
... we have 28 MTI stats that show as not implemented in WPT, but probably just because they've moved recently and the tests need to be updated

Youenn: Firefox and Chrome are the two main implementor for stats

Armando: does the tests need an update?

JIB: no, but implementations need to catch up with stats
... I don't think they should be in jeopardy because they moved
... Firefox doesn't have track-stats yet

Dom: is the list of MTI stats in -pc correct?

Henrik: it will have to be updated to match the results of our stats discussion tomorrow

JIB: OAuthCredentials?

RESOLUTION: move OAuthCredential to extension

JIB: negotiate for rtcpTransport? (issue 3 in spec)

Harald: impl got nuked, too confusing

RESOLUTION: negotiate for rtcpTransport to be removed

JIB: VoiceActivityDetection?

Henrik: over taken by events, let's remove

RESOLUTION: remove VoiceActivityDetection

JIB: getDefaultIceServers()?

Youenn: move to an extension and wait for a proponent to push for an improved proposal

JIB: getSUpportedAlgorithms?

RESOLUTION: remove getSupportedAlgorithms

JIB: SendParameters.priority?

Orphis: remove

RESOLUTION: remove SendParameters.priority and prioritytype enum
... remove ReceiveParameters.encodings

Orphis: (they're pointless)

JIB: EncodingParameters.dtx?

Bernard: nobody implements

Orphis: available via SDP munging for OPUS in Chrome

Harald: let's drop it

Orphis: this applies to ptime and codecpayloadtype should also be at risk

Henrik: a large discussion: there would be a need to control the codecs you sent beyond what to do with negotiation today

Orphis: right now, it's read-only

Henrik: codecpayloadtype could be removed and brought back later if there is demand in an extension

Orphis: ptime isn't implemented, not clear demand

RESOLUTION: Remove .dtx, .ptime, .codecpayloadtype from EncodingParameters
... datachannel priority to be removed

Dom: Identity - mozilla sole implementor?

JIB: we want to see it adopted
... the peerIdentity steps could perhaps be moved to -identity

Henrik: moving out is an editorial question
... but the real question is whether identity is part of 1.0

Youenn: no plan to implement in 1.0; interested in the field, but not sure this is the right solution
... there may be other issues that need to be addressed with higher priority
... we don't want to be tied to that particular solution at this time

RESOLUTION: migrate the remaining identity steps to -identity

Dom: we also need to figure out how to re-integrate isolated media streams into our normative sphere

Developer feedback

<inserted> ScribeNick: dontcallmeDOM

Mészáros Mihály’s Developer Feedback

Mészáros: TURN for NRENs, Higher education & research.

scribe: Global turn server, the goal is to keep media traffic in their network.
... Multi-tenant TURN is not possible without OAuth + TURN.
... Working on co-located TURN, working on implementation in Firefox and possibly Chrome.
... Why not just add these three attributes, it would require little changes to mechanisms. Should be little effort.
... We should consider keeping OAuth until we have alternatives for TURN.

Dom: Pressure to reach 1.0. Moving this out of the spec does not mean it is not going to happen, but we will need people to get involved to make sure spec and implementations are maintained. And write tests. We will likely reach out to you about future involvement.

Youenn: You say the existing API is not great, but is the current API working as a hack?

Mészáros: We use Time Limited Long Term Credentials and the behave TURN REST draft but I am not happy with this, there is a better way. We rely on the non-standard (https://github.com/juberti/turn-rest). TURN REST discontinued to move on OAuth. LTC (username + password) not designed for browsers, a browser can't keep secret in JS the password..

Harald: Do any of the browsers implement the behave TURN REST? How do you do this with current implementations of browsers?

Mészáros: TURN REST from browser side is the same as LTC, no implementation required. LTC is not designed for browsers.. It would be nice to have one authentication type that is IETF Standard and designed for browsers. OAuth credential is time limited and scoped to a server, so it is more secure. Please keep it in the Spec!

Sean’s Developer Feedback

Sean: I work on a go implementation of WebRTC. A lot of people come with complaints about how they want their ICE to work. There’s so much going on.
... Small businesses want to use WebRTC but they have to use other things because they have restricted port ranges.
... Once a week someone complains about addIceCandidate before setRemoteDescription. Can we catch them or throw them in a queue? Re-order operations? It would save new developers a lot of time.

Jan-Ivar: The reason you can’t do addIceCandidate in a different order they could end up in the first offer.

Sean: That’s only an issue for the first offer.

Jan-Ivar: There’s no limitation you can only negotiate once. There’s a race potential: If you get a remote offer you have to set it before you await other operations.

Sean: In the real world this is a problem. People cache candidates for example. Jan-Ivar: You shouldn’t cache.
... Closing (message+code) can be done by sending a final message
... Ability to deny a data channel based on data channel name. Not sure if enough value.
... Media via DataChannels is being more and more popular. Lack of HW acceleration.
... Some people would be happy with more latency for less packet loss? Both reliable and unreliable data channels. People want more flexibility than they can get with the current APIs.
... People want setCodecPreferences; SDP munging is painful. I’m tired of seeing people munge SDP and it blows up in their face.
... Lack of CipherSuite choices: people want HW acceleration.
... addStream vs addTrack vs addTransceiver. Causes headaches, people don’t understand what’s going on.
... Provisional offers/answer: Do we need them? I have to answer questions about it all the time.
... Can we make API simpler for porting to other languages like not having to use strings?
... Tor Snowflake uses DataChannels and spend a lot of time removing the ability to fingerprint. Can we set ciphers?

Silvia’s Developer Feedback

Silvia: Mostly complaining about things from a WebRTC Telemedicine developer’s point of view. We have to jump through hoops to make things interoperable! Chrome, Firefox, Safari.
... We have to recommend people not to use Firefox due to worse media quality. High-level report; don’t know why this is the case.
... We use the new API, using multiple streams works well in Chrome but not in Firefox. Interop problems.
... We sometimes hit decoding problems.
... Stats is a pain point: new getStats has much less information than the old one. Plus different browsers have different implementations of the standard getStats API as well.
... The API doesn’t really tell you whether or not media is flowing. We look at time of the media elements, usually it works but not always. It’s hard to detect playback issues. Any suggestions on finding out if media disappeared? The browser thinks the connection is there but we’re not receiving any frames.
... Data channels for video: You don’t have to deal with a lot of complexities. Interoperability is combinatorially more complicated when you have to deal with all the different versions of the specs.

Emil’s Developer Feedback

Emil: Will put in slides after the meeting. I have three items:
... One of the pain points: messiness around switching “plans” (Plan B vs Unified Plan). This does not solve any problems for us; it would be good to ensure this could be done gradually. Like SSRCs vs RIDs: it would be good to have SSRCs in the meantime while SFUs upgrade to RID support.
... We keep hearing not all windows are sharable on windows.
... As an SFU provider we want to assist call quality. Take audio quality for example, there’s really no way for us to improve audio quality. For video it’s simpler but limited. We would like to be able to add our own FEC to PeerConnection. Those hooks discussed a few years ago by Justin would be really nice. We would like to do End-to-End Encryption. Harald’s proposal is compatible with our goals. We would like hooks on the client side at diff

erent points in the pipeline to add our own logic.

“Others on the Line”: Max

Max: Lack of granular port ranges.
... How do we add hooks on different parts of the stack. We are excited about these things becoming part of the standard and
... Currently stuck on Plan B and is experiencing difficulties shipping Unified Plan, especially around SSRCs vs RIDs, but plan is to move to MID/RID multiplexing architecture.

Harald: Do you think we should revive SSRC signaling? Should we revive the draft that I wrote?

Max: Yes. Loudly adding a plus one!

WebRTC TPAC F2F Day 1

19 Sep 2019

Attendees

Contents