06-07 Nov 2017



Bernard, AdamBe, DanBurnett, Huib, PeterT, JanIvar, Varun, Cullen, Dom, hirokias, hiroki_asakimori, Stefan_(remote), Jan-Ivar, Youenn, EricCarlson, Hyukhoon_shim, PeterB_(remote), Soares
Bernard, HTA (remote), Stefan (remote)
dom, pthatcher, burn, DrAlex_, DrAlex, fluffy, adambe, vr000m, soareschen


<hta2> I don't see anyone on the testing webex - are you in the meeting webex?

<fluffy> stefan, are you on IM here ?

<dom> hta2, can you say something?

<fluffy> IM us back as we are not hearing you

<hta2> Sorry, not looking at all the screens all the time while waiting - did you get the sound check for me OK?

<inserted> ScribeNick: dom

WebRTC Testing

<hta2> Sound from the meeting room is very low.

<lgrahl> Ouch!

<hta2> Sounds like you have too many devices with the loudspeaker on in the room.

<hta2> if someone can turn the camera to point at the speaker, that might be nice.

Huib: I work for Google; used to work for Opera and have been working with Web technologies for a long time
... have been working on finalizing the WebRTC technology and we have been working with testing to that end
... making the Web work reliably is a key aspect of providing APIs ot the Web
... WebRTC 1.0 has been a long road, with many specs both APIs and protocols at W3C and IETF
... lots of efforts to get there
... Web Platform Tests is a great tool for testing Web technologies, but it is not sufficient for WebRTC given the complexity of WebRTC, and the number of underlying low level protocols
... So Google took up the idea of Alex to work on a more comprehensive system - KITE, an open source project we formally announced last week
... Alex will tell us more about it
... Beyond spec and testing, the third part if compliance - all browsers have their own challenges in that space
... We've made progress in Chrome in the past year - compliance on getStats, constraints, RTCRtpReceiver
... The major upcoming milestone is unified plan, planned for Q1 2018
... getting to 1.0 is critical to avoid getting too many proprietary or native solutions deployed for WebRTC
... The major engineering effort in WebRTC in Chrome is around reliability - which is much harder than spec compliance
... other browsers are probably in the same situation
... Soares will report on his work on WPT, sponsored by Google

Soares: quick update on the status of compliance testing in WPT
... We have ~1300 tests - added 1000 in the past few months
... We have added a tool to calculate coverage for WebRTC

<hta2> Repeat: If it is possible to turn the camera towards the speaker, that would be good.

Soares: We extracted all the normative statements

<hta2> Thanks!

Soares: the WebRTC 1.0 spec is about 70% tested at this stage!

<lgrahl> Can you talk louder? Sound is cut off a lot.

Soares: Some of the challenges to reach full coverage
... tests are currently written based on the June editors draft - quite a few updates since then, so the tests are a bit out of sync
... some race conditions make test difficult
... WebRTC describe steps that happen in parallel - sometimes hard to trigger specific concurrent operations

DanBurnett: you noted 10% untestable assertions - what's your thinking on that?

Soares: the untestable ones are bound to the constraint of what is exposed in plain JavaScript

DanBurnett: if these assertions come from the spec, the spec may need to be changed to avoid those

Alex: Some errors can't be dealt with at the JavaScript level - e.g. errors linked to hardware error which can't be triggered
... re race conditions, dealing with timing and promise-based algorithms is hard

Jan-Ivar: we may need to fix the spec if the timing is not deterministic

Alex: re untestable, we count them as "tested" in the 70%
... regarding the new 97 steps added since June, many of them come from error reported out of the testing effort
... in the remaining path toward full coverage, 90% of it is easy; we expect to reach 100% in Q1 2018
... once we reach 100% for webrtc-pc

Soares: note that coverage doesn't mean they are all correct - the tests need to be verified by implementors and editors
... one possible topic is how to keep track of coverage over time
... we're looking at automation with web driver & permissions API
... there are other specs that need testing efforts: we are on track on testing mediacapture-main next - other specs are further down in the pipeline

[Slides: Real-Time Communication Testing Evolution with WebRTC]

Alex: beyond testing the JavaScript compliance, there is a lot more that is needed to properly test an RTC system
... JS API compliance is part of the standardization process - but the tests are standalone tests, testing a single browser at a time
... Until WebRTC, this was probably sufficient
... but for WebRTC, you need tests that test interop across browsers
... you also need to test intersection with protocols and the networking layer (e.g. ICE, NAT traversal)
... So beyond compliance testing (with automation, stand alone tests, one browser), we need to move to interop testing (still with automation, but on two synchronized instances of the test, with 2 or more browsers)

<lgrahl> Sorry for nagging but sound is cut off a lot again. :/

Alex: For automating interactions, Web Driver helps quite a bit
... But we need improvements there - e.g. extensions to integrate permissions management in Web Driver
... Once you've obtained permissions, you need ways to emulate captured media
... browsers have already ad-hoc ways to do that, but we want a standardized Web driver approach to it
... So Web Drivers help quite a bit, but it doesn't help when it comes to run synchronized tests across browsers
... lots of tools hvae been developed to address part of the problem - which confirms the problem exists and is important
... how bad is it when you try testing interop across browsers / OS - leading to ~500 situations to test
... only very few of these situations had been automated so far
... existing solutions have limited reach in terms of browser versions and OS
... eg in the enterprise market, Windows 7 is still a key OS to test for
... different browsers have different level of support for Web Drivers, and even more so when taking into account the permissions extension
... to avoid regression tests or ensure newer versions of browsers are as compliant / interoperable as possible, bugs need to be caught early in the release process (i.e. in ~nightly builds)
... depending on resources, your needs on freedom/convenience/price vary quite a bit
... Taking all these dimensions into account, what you can achieve with existing solutions is always limited
... Testing WebRTC feels like death by overwork, hence we created this interop testing engine named "Karoshi Interop Test Engine", aka KITE
... it is designed to be as generic as possible
... meant to work with any client, any platform, any back-end
... we architected this by designing 4 independent components: a selenium grid, a test engine, tests themselves, and a reporting dashboard
... tests and dashboard aren't completely independent since the dashboard needs to know what to expect from the tests
... Tests have to encapsulated into a Java class
... configuration files are used to describe which tests are run in what browsers, on what selenium nodes
... we also record timing information to detect performance issues

<lgrahl> Sound (sorry) :)

[demo of KITE dashboards and how they allow to look into details of runs]

scribe: This level of testing has already allowed us to identify and report bugs
... we have plans to extend this to mobile browsers (will come to open source project), to electron and other native platforms (in the cosmo proprietary version)
... we will bring additions for SFU testing over time
... we are very open to all suggestions
... we want to make this tool as useful as possible - please file issues on our github repo

Cullen: first, thanks, this is amazing

<hta2> the sound has become worse and worse over the period of the presentation, with mike cutting out randomly. Now Cullen's voice is much better than Dr. Alex' was.

Cullen: what are the next steps to expand this

Alex: Web Driver handling is key; support in Edge is important to reach 100%
... Tests for IETF protocols are a big piece of the picture

<hta2> the new mike sounds better, at least at first.

Alex: a key topic for people is call quality

<hta2> but now it's doing the same thing again.

Alex: media quality is another topic - but hard to measure automatically
... What I want is have a tool that moves beyond compliance to quality, incl network and media resilience
... As far as standardization is involved, the most difficult part will be around the number of protols

Cullen: the key is identifying key use cases - it's impossible to test all the situations covered by the RFCs

Huib: from Google side, we've sponsored the development of KITE to enable the proper testing of interop testing
... getting the important use cases exposed is key
... we want the baseline scenarios to be right

dom: how do you write an actual test case?

<lgrahl> The mic seems to work decent if the person talks loud.

Alex: we need to improve the documentation on this; this is upcoming in the next few weeks

dom: how hard/easy is it to run WPT tests in KITE?

Alex: we're looking into making it easier

<hta2> now the sound is completely gone.

Huib: but this not the main use case for KITE - the expectation is that this will run in the WPT dashboard

<lgrahl> Thanks dom for summarising - sadly I couldn't hear much. :)

[coffee break until 10:45]

Welcome / Intro

[reviewing slides]

Test as You Commit Policy

Adopt test as you commit policy

Philip: I work from Stockholm Google office on testing
... WPT is big
... WebRTC test suite is in pretty big shape
... To keep the test suite in good shape, I have a proposal to keep them in sync as the spec evolves
... We have an annotated version of the spec based on test coverage rwaldron.github.io/webrtc-pc/
... the coverage of that spec is one of the better known of the platform
... The WebRTC test suite is run daily on the WPT dashboard wpt.fyi
... currently has lots of red, but most of it is linked to automation of getUserMedia (or lack thereof)
... both for permission prompt and fake media generation
... so this should get much greener soon for Chrome and Firefox soon
... we have a new piece of infrastructure to have tests to drive themselves via Web Driver
... we plan to extend the Web Driver API to manage permission prompt and to bring that to the WebRTC test case
... the WebRTC test suite is being imported into browser test suites (at least Chromium, Gecko, Webkit)
... also test case updates get automatically upstreamed
... My proposal today relates to an approach groups have started to adopt
... where a normative change to the spec comes with an associated pull request on the test suite

<pthatcher> Sure...except my slides are next.

Philip: we hope to increase the production of tests based on that policy - this is already having a clear impact

<pthatcher> Also.. .when I do scribe, do I have to be as detailed as you? I don't know if I'm capable of typing that quickly.

Philip: we have passed 50% of specs who have adopted the policy

danburnett: there is a discussion in the issue - e.g. what to do if you don't know how to write test case
... also, we should understand if/when that policy applies - for CR, it makes plenty of sense; for new work, there may be too much change early on

Philip: the proposal here is to do this for WebRTC 1.0 spec
... some groups have decided CR is a good time to enforce this
... but clearly it can't wait until CR

Cullen: at a conceptual level, sounds great
... the practicality of it is a bit worrisome - will it slow us down in closing bugs? it will improve quality for sure
... but maybe before adopting it, we should play with it to see how it works out

Philip: note that the policy is not about adding a hard requirement so much as encouraging it as a practice
... this may slow down the spec work, but would increase the rate of interop

Youenn: fixing the platform is more important than fixing the spec
... this simplifies implementation in browser vendors

<inserted> ScribeNick: pthatcher

Dom: how many people have experience writing tests cases, among people who make changes to the text?

Cullen: I don't

(a few hands saying they do?)

foolip: the person who wants it merged is going to want to see the test added
... but the test can be written by someone else

dan: Idea for policy: for a PR to be merged, there must be a test. But the editors can make a judgement call to be able to merge without a test.

foolip: This is like tests in a code review. It's a cultural change.

Harald: feels safe to try this;
... if we don't try it, we won't get there

jan-ivar: What's the granularity?

Harald: I'm just talking for volume testing purposes

foolip: ... Can't be adding huge new things; so shouldn't need huge new tests for small changes
... just small tests for small changes

dan: maybe we should wait until we have a certain test coverage threshold

Bernard: a call for consenus on issue 1617 to provide a test for changes unless you convince the editors it doesn't apply to you

lots of thumbs up

no thumbs down

RESOLUTION: We adopt a test as you commit policy as describe in issue 1617 - we may revisit if it proves too complex in practice

Noted: a consensus to move ahead, with Cullen's coveat that we can revisit this if we find it doesn't work well

foolip: I'll go write that PR

I can't scribe any more :)

WebRTC-PC Issues

<dom> ScribeNick: dom

RTCPriorityType combines relative bitrate with QoS priority, which applications may not want

PeterT: we discussed as an interim before
... right now, you can't control separately priority and QoS

Adding relativeBitrate parameter to RTCRtpEncodingParameters

Peter: the PR provides a more granular control on RtpEncodingParameters, splitting bitrate and QoS priorities, with a more flexible control on bitrate

Varun: relativeBitrate is that related to what?

PeterT: to other streams

Varun: so the top bitrate is one?

Cullen: there was confusion about rate being used and max

PeterT: same as currently, but with more control on the relative values

Harald: I worry about this
... what you're saying now is that if the congestion point is at the point of having DSCP glitches, you won't change anything
... e.g. if you want to prefer audio over video and get into congestion, the relative bitrate will not get you what you want (e.g. throwing video)

petert: if we focus first on splitting qos vs bitrate - what's your opinion?

hta: from a process perspective, the IETF specs have already passed last call, so that's a concern
... from a practical perspective, the risk is that we open up many cases that make no sense for users that wouldn't do the right thing

PeterT: there are 2 type of users: those that don't care and those that do
... having one knob helps the former, having two helps the latter

Harald: the relative bitrate control seems useless to me; but changing the spec of control we already agreed upon worries me

Bernard: exploring one of the cases that match this question: audio conf with low priority file transfer; but there is no reason to keep its bitrate low

harald: the current spec says that when there is congestion, the low priority @@@

stefan: we should have much more data about actual usage nowadays
... I would like to see proof of need of this
... to get data on how this actually behaves

PeterT: we have an application that wants to use this field
... but when trying to implement it, the fact that the two are combined was blocking us

Varun: we came across this as well

PeterT: we want to turn on DSCP markings to experiment with them, but we can't do that if this is coupled with bitrate

Cullen: assuming we would agree to do this (i.e. it's not useless), what would we need to change in specs?

<foolip> who wants to merge https://github.com/w3c/webrtc-pc/pull/1657?

Peter: so imagine we have one 4 level knob for QoS, and a 4 level knob for bitrate

Cullen: I don't think such a change would have to impact the RTCWeb-transports draft

HTA: RTCWeb transports is where 1/2/4/8 is defined

PeterT: I'm happy with only doing the split of knobs - the relative bitrate thing is just a nice to have

<hta2> https://tools.ietf.org/html/draft-ietf-rtcweb-transports-17#section-4

Cullen: I'm not against it; I'm not sure I'm convinced yet

PeterT: so the model would be to keep the current knob, and add two "experts" one for separate control that would override the other

Cullen: splitting the knobs would impact the rtcweb-transports draft

PeterT: Let's look into this offline and come back to this tomorrow

Issue 1635 Need for Initial Bitrate by the Application/RtpSender?

Need for Initial Bitrate by the Application/RtpSender? #1635

Varun: the IETF congestion control wanted lots of info to help with initialization

Cullen: I think the IETF would have concerns about this being misused by attackers in JavaScript

Bernard: RMCAT could send a liaison statement if they want to push for the W3C WG to adopt this

PeterT: my recommendation was pushing this for post 1.0

Varun: that's fine by me - but I want to emphasize that people are complaining about the lack of current optimal starting situation
... I'll reach out to the RMCAT WG to let them know they should a formal request

PeterT: but even if and when they do, it's not clear where we would put it

Bernard: we'll put that issue on the icebox for now

<fippo> fluffy: won't those security concerns apply to what is currently possible in chrome with sdp munging?

Issue 1194 audioLevel of tracks, both send and receive?

audioLevel of tracks, both send and receive? #1194

PeterT: this can be done with Web Audio

Cullen: but not when the stream is isolated

PeterT: recommendation is not adding this

Bernard: no objection

Adding more values to RTCIceTransportPolicy Enum #1644

Adding more values to RTCIceTransportPolicy Enum #1644

PeterT: 2 different issue - one about network permission separate from getUserMedia, the other about giving more control for the app to constraint their routing

Varun: use cases including routing data through or not VPN
... another thing is to enable a relay-first approach to help with establishing a call quickly

Dom: so if we don't do this, what should we recommend to the OP?

PeterT: so for the first bit, we need to look into the permissions thing
... for the second, give more justification / use case

Cullen: maybe a way to address the second thing is a way to expose from which route the origin was loaded

PeterT: it may make sense to close this issue and open one on data-channel only permission, and then maybe one discussion exposing more routing info for selecting ice candidates

behaviour of offerToReceive* set to false when there is a local track #1586

behaviour of offerToReceive* set to false when there is a local track #1586

<fippo> i probably owe a PR here. Its a mess. (dom: note that so its on record :-))

JIB: this case would only happen in the case where one of the party is not a browser

RESOLUTION: we don't change how offerToReceive*: false behaves in the spec

Isolated Media Streams requires modification on permission algorithms in GUM and Permissions specs #1646

Isolated Media Streams requires modification on permission algorithms in GUM and Permissions specs #1646

Soares: discovered thing while testing isolated media
... right now, the way the permission grant doesn't take into account the fact that permission is granted not to an origin, but to a peer

DanBurnett: this needs to be described in webrtc not getusermedia

Dom: but getusermedia probably needs a hook in its permission check
... so I guess we do need to modify the specs

Soares: I can take a stab at it

Alex: who will review?

Dom: DanB+JIB for getUsermedia, MartinT & EKR for webrtc, …

SVC Use Cases For an SFU developer

Bernard: this is input to our discussion tomorrow on WebRTC NV
... we're doing it today due to time constraints of the presenter

Sergio: this is meant as a conversation trigger for WebRTC NV

[slide SVC Use cases]

[slide Content Adaptation (II)]

Cullen: decoding complexity seems a bit different from the rest
... e.g. based on codec support
... e.g. it might be that stereo videos would use different layers for the 2 eyes

Sergio: the most clear need is turning on/off temporal vs spatial scalability

PeterT: I think we have everything in RtpEncodingParameters, except for the spatial/temporal switches

Varun: there is also the question of priority - which layer should switch to which priority
... not entirely sure if that level of control is needed

Cullen: I agree with these requirements
... when it comes to image size, we care about the thresholds
... we code to specific sizes for specific phones
... right now we work around this via stats
... but it would be more convenient to have direct control over this

DanB: This all looks great; but how doable is it? how standardized are the controls across SVC codecs?
... every codec seems to be doing it a bit differently
... can we define a predictable outcome?

PeterT: I believe if we just add the high level switches for scalability (temporal and spatial), yes

Sergio: agreed; it would be hard to do low-level control because of this variability, but high level control seems very feasible

Bernard: there is an implicit assumption in what PeterT said: that a decoder can always decode what an encoder sends
... that's true for VP9, but it may not be true for all current, let alone future codecs

PeterT: if you set the scalability bit, if the codec doesn't support it or if the recipient can't support, then you just use simulcast

DanB: that's what's at the heart of my comment - can this actually be implemented?

cullen: there aren't so many SVC codecs, and they've all been designed similarly - so yes, this should be doable

Bernard: AV1 has a similar syntactic structure simliar to H264
... it may break the assumptions of VP8/VP9 of encoder // decoder

<burn> scribenick: burn

WebRTC Stats

[Varun presents slides]

<dom> WebRTC Stats Issues (30 atm)

Issue 99/PR 251: Security and privacy considerations

s;Issue 99/PR 251: Security and privacy considerations;Topic: Issue 99/PR 251. Security and privacy considerations;

Dom: for the security-impacting features in Stats that are beyond what webrtc-pc itself introduces, we will send the APIs to the TAG with a proposal

Bernard: "network layer data" is vague. I suggest describing "statistics related to packets lost/received, etc."

Varun: counters might be better

<dom> Cullen: some of the stats exposed here can fingerprint a specific network based on its characteristics, which can help determine two peers are on the same local network

Varun: regarding location access, we need feedback

Cullen: what can you do with this that you can't do with HTTPS?

Dom: by exposing rtt between peers, given the location of one you may be able to infer location of the other

HTA: can tell distance between clients, not just clients and servers

Dom: and these stats can be used without the user knowing it. Data channels alone with this could be used to triangulate.

Varun: and this is useful for CDNs.

Bernard: candidate pairs also give you RTTs.

Dom: we will expose the problems that exist and discuss possible mitigations

Next step would be to merge this PR (after Varun updates).

Issue 177. Caching and consistency of getStats()

<dom> Discuss caching and consistency of getStats() return #177

Varun: PR 243 is related

JIB: PR says when you cannot cache

Dom: since this says For Example, the MUST should be lower case

Issue 131/PR 262. "objectDeleted" marker

<dom> Consistent marker for "non-active" object? #131

<dom> Added 'objectDeleted' attribute #262

JIB: if we keep around stale data, under what conditions would they go away?

<dom> https://github.com/w3c/webrtc-stats/issues/235

Varun: this came up before, but we haven't seen any problems with it. Issue 235

<dom> Is keeping stats around a memory problem? #235

Varun: we haven't seen it, but others might.

JIB: Why can't we just grab stats just before closing an object?

HTA: sometimes an object is destroyed by remote control, e.g. setRD that closes some of the connections.

Varun: also a third party might be monitoring and not be privy to the actions taken by the application.

JIB: what about a time limit?

HTA: for a long time now the limit has been the life of the PC
... we could add an API for stats clearing.

Adam: what about an API to register a capture for anything you would care about?

Varun: if you are a third-party monitor this is too hard to make work

JIB: this is a slippery slope. Maybe we need something else in reality, like a session ended event. Let's solve the right problem.

HTA: we have event transitions in theory. Maybe in NV we can consider other options I will send by email.

JIB: FF doesn't work like this, so we want to make sure this is necessary before we do it.

Dom: does that mean issue 235 should be a CR blocker?

JIB: we will review and get back to you.
... this is related to sender vs. receiver stats

Varun: back to 262, should we do it?

JIB: that's where we had concerns

Varun: if 262 depends on 235, which is in icebox, we have a problem

Dom: if either blocks CR we need to discuss.

JIB: yes, we need to review and propose an alternate

Dom: are we marking 235 as CR blocking and linking it to this issue?


and Jan-Ivar owes a response to it.

Issue 231/PR 273. Sender and Receiver stats vs Track stats

JIB: using replaceTrack will replace the counters on the sender side but not on the receiver side
... builds on next topic of Issue 230/PR 272.
... idea is to have counters on senders and receivers.

varun: not having track info there is a problem with replaceTrack because track stats go away in this approach. We need the stats to stay around.

JIB: this sounds CallStats specific. In general we have assumed that with a PC we don't get a done event for every action.

Varun: intercepting WebRTC APIs is problematic.

JIB: this is not an issue for original JS app code, since their code causes the actions and thus knows.
... instrumentation is an edge case, albeit an important one.
... how often do we need to know the counts before and after a replaceTrack?

Varun: switching front to back cameras, or screen capture, are very good examples.

JIB: getting tough to parse stats now, since you have to know which objects have been deleted.

Varun: we should separate the two issues / the two PRs. Getting an event would be nice, but we hear users asking to be able to call getStats at the end, after objects have been ended.
... this has not been an issue so far because we haven't had lots of tracks lying around. Hard to know how this will change going forward.

JIB: the issue seems to be removing track stats, right? Is there a problem with adding the sender and receiver stats?

varun: not at all. It's the removal of the track stats that would be an issue.

JIB: okay, let's focus on the addition first.

varun: +1

JIB: I will update the PR

Issue 230/PR 272. RTCMediaStreamTracks to 4 dicts

JIB: this is editorial only since the API is unaffected.

Varun: no objections. What would be required now if we get depth stats?

JIB: same as what we have. Might be a bit more prose.

Cullen: how does extensibility work?

Varun: easier with an enum.

JIB: depth is simpler because it inherits off the existing dictionaries

HTA: another extensibility piece to track.

JIB: any objections?

Varun: let's make sure this is JS observable.

(no objections)

Issue 133/135. DSCP information

Cullen: (asks about how it would look)

HTA: we have two alternatives because it's not really clear why someone would prefer one over the other

Cullen: maplike is easier to use, but a string could be parsed as well.

HTA: (missed comment on strings). With maplike you would have to iterate.

Dom: any privacy or security issues?
... say, bleaching of JS?

Varun and Cullen: yes, but there's not much you know.

JIB: record is closer to a JS object so might be passed by reference, but don't know for sure.

(agreement this is useful)

(but not on which to choose)

JIB will review and comment

Issue 202. concealedAudibleSamples

Bernard: why during audible portions?

Varun: if concealment is noise or content is important.

Cullen: there is an RFC that defines concealed.

Bernard: in XR block

Varun: but it doesn't say whether it's in an audible section or not.
... packet loss, audible sample, or not are the three factors

Bernard: RFC 7294

Varun: there are many issues like this that are not controversial but are lacking expert review.

Cullen: we were only supposed to reference existing stats and not create new ones, so maybe these need to be removed if they are unclear.
... we were only supposed to add stats that are clear to the spec editor.

Dom: and this is not a CR blocker? why not?

Varun: virtually everything in getStats is optional anyway. We may just provide a date cutoff for something going in.
... as soon as all the CR blocking items are in we will move on even without these iffy items going in

Dom: should warn people of what might go in/not go in.

Varun: conclusion is we need a citeable reference that all parties agree on.

Dom: Cullen is right that the experts are not likely in our group.

Cullen: this specific metric has problems.
... the concealment algorithm introduced audio, but that's about all. Get IETF to define the details if you really want it. Concealed seconds is useful, but the high/low energy distinction is of questionable use.

Varun: this is in a grey area where I don't see the value but that doesn't mean it isn't useful to someone else.

hta: we have lost samples that occur in the middle of speech, so you might want to replay rather than filling in.
... typical suggestion by someone but we can't decide whether to refuse or find a better definition.

burn: the burden needs to be on the suggester.

Dom: let's ask for a more specific justification and reference.

Issue 1613. Stats and Isolated Streams

<dom> Stats and Isolated Streams

HTA: we know this can be an issue. So either we try to mitigate, or we just remove the stat. The latter is my suggestion.

Cullen: security arch doc actually says to do that.

HTA: I will prepare a PR for not reporting the stat in this case.

Bernard: an issue for csrcs as well
... in webrtc-pc

Dom: will the PR address CSRCs as well, or just volume in isolated streams?

HTA: i will do the stats part.

Dom: I will file a new issue for CSRCs

HTA: I will do the PR then.

Media Capture and Streams

<dom> ScribeNick: DrAlex_

<DrAlex> jib speaking

<DrAlex> starting with jib tickets in the absence of peter

sometimes the values that were preset are different from the current values

example, capture rate might fluctuate up to a preset of 60 fps

live volume vs actual volume settings

is another example.

camera motor pan has a target, and a current value as well.

cullen: let s take th pan example

we need both values

jib: yes but one is not a constrainable

cullen: agreed

dan: we already discussed this

this should be what it is SET to

hence the name.

we can still discuss wether we also need a current value or something, but the result of getSettings() should be what i was SET too

bernard: the downside is that if we had a current value, people would start pooling it.

jib: the spec has special frame rate wording.

dan: there may be a precision discussion we may have across

all measurements.

that is also true for framerate

which can fluctuate around a target velue

but your setting is what it is, and doe snto change.

cullen: it s ok to report error if you make a mistake.

or if there is a problem.

you should not set constraint, but when you do, and use exact, you are expecting errors when things is not exact.

camera rates do not fluctuate. However browsers get frames at different rates.

jib: cameras, e.g. on mac, have modes. if you ask for an arbitrary rate you night not get it.

it is my opinion JS should count the frame rate themselves.

cullen: i do not think there is a way for app to compute FPS, really.

jib: ok, so for motor pan .....

do we speak about target, settings, values


cullen: actually: what do we have again ?

<inserted> ScribeNick: DrAlex

issue 466

the answer is "it shoul dbe whatever the last setting was"

for previous issue.

<stefanh> was there any conclusion on 470?

<stefanh> audio is really bad

conclusion of 470 the answer is "it shoul dbe whatever the last setting was"

<dom> (i.e. the target setting, not the live value)

settings shoul now belong to the track

<stefanh> +1 to hta's comment

of course source have settings, but source can have multiple settings, from the user point of view (and also for security reason) the settings should be on the track


it would be very easy to see the track as a rescuer object.

should the track only reflect the source settings

or shoul dit handle all the scaler.

dan: when we spoke about it initially, we had decided that the sink would be the one holding the processing (scaling here)

jib: we have now additional sink: canvas capture, remote track .. we have never came up to define constraint on those. e.g. echo cancellation is audio only .....

with that, I think that if we can word this in such a way that tracks can have different settings we are still all in agreement

cullen: isn t it like that already ?

consensus on leaving it as is

<DrAlex_> harald: we decided last time we had two ways to do it.

<DrAlex_> one sends nothing

<DrAlex_> one sends black frame.

<DrAlex_> cullen: while I think we should specify it so people know what to expect, but it s an implementation detail.

<DrAlex_> sergio: in the case of simulcast

<DrAlex_> we also have that kind of problem, when we kind of "mute" the higher resolution in case of congestion. In the IETF there is a solution.

<DrAlex_> bernard: i just looked it up. there is nothing about enabled=false

<DrAlex_> ericsson: it does not matter if it s muted or enabled=false, it should be the same behavior.

<DrAlex_> <everybody agrees>

<DrAlex_> jib: so ... no action item ?

<DrAlex_> cullen: two different settings. one is resulting in silence / black to be send the second one stops rtp packets.

<DrAlex_> bernard: track=null results in no rtp sent.

the track could be enabled, while the source be muted.


adam: the specs says balck frames and silence shoul dbe send. but it does not specify wether you send only one such black frame, one per second, or full stream of black frames at 30fps.

youenn: safari sends one frame per second, just to make sure something is coming.

so does firefox.

<audio setting in the room>

jib: do have a motion toward consensus on this.


495 / 472 Video dimension constraint

<stefanh> What was decided on 441?

441: one frame per second.

scenario: sender has 16:9 receiver wants 1:1

<stefanh> BTW, the "one frame per second" was an advice, right?

shall we add pixels (letterbox) or removing piixels (crop)

proposal: new constraint resizeMode

what a bout default

cullen: ok, what abou tnone?

peter: not allowed to do any resizing

cullen: ok, love it.

peter: there is still the question of default

cullen "trash my video"

jib: my one concern is that we are moving to a "rescaler API" we argued shoul dbe on the sink. if we re speaking about access to the source ......

i like crop'n scale

cullen: i originally though like you. but justin convinced me that for screen sharing cannot let any pixel to be thrown.

jib: different spec

adam: surveillance video also need to keep all pixels.

jib: ok

peter: to you point that it s become a size and rescale API, yes, and that s what we aim at.

Dan: i do not think it conflicts with anything else.

dom: one confusing thing, there are many different constraints.

<stefanh> FWIW I like Peter's proposal 'resizeMode'

dom: we should start making the distinction between the constraints used through applyConstraint, and those used in GUM, ....

<dom> objectFit

<dom> +1

Harald: we should reuse the names from CSS

peter: agreed. i just could not remember them when i wrote those slides.

harald: i ll dig them up

jib: so ....

the implementation concern is not w3c"s priority, but .... for browser vendors, there are risks.

peter: this should ONLY be used for aspect ratio.

if the aspect ratio does nto change, it is not intended to be used.

<stefanh> my understanding was that resizeMode would be used also when width/height was constrained

jib: we might end up rescaling twice then

peter: yeah i guess

jib: do we want a 4th value "preserve aspect" as per cullen suggestion?

peter: i ve never heard anybody asking for that.

jib: any other comment ?

consensus in the room

dan: the conversation was going fine, so i was DL the audio system info

jib: is there a pr

<dom> consensus to add a resizeMode constraint

peter: no, not yet

<dom> JIB will produce a PR

<stefanh> consensus on default mode?

jib: ok, i ll write the PR.

<stefanh> crop-and-scale?

jib: with the new constraint we have a way forward, but we still need to define a default.

cullen: constriant has a mechanism: minimize a istance function

youenn: no scaling at all, so we would prefer minimize processing.

jib vs cullen round 5

on constrint.


proposal: unless people specifically requires resizeMode, the default is to let the browser choose


<dom> Chair: Bernard_Aboba, Stefan_Hakasson_(remote)

<dom> Agenda: https://www.w3.org/2011/04/webrtc/wiki/November_6_-_7_2017

<dom> Slides

<dom> ScribeNick: fluffy

Content hints for MediaStreamTrack

On slide for "Issue 478"

Questions abotu what to do. Not many questions asnwered since last time

This would be a new field on track or something like that

Seems where we are is people previously thought this was generally a good idea but wanted to see specifics before they agreed to accept or not

Argument for putting it in a track was that it needed to go to multiple things, like recorder, so that made it better in track and sender

<dom> cullen: another question is whether we have enough and the right categories

<dom> ... e.g. having 30 (sport, ...) vs 2 modes, this is very different conversation

Main driving use case was indicate that video should be treated like screen cast

Question does this go on tracks or sinks

Concerns raised about auto setting as this implies it is a hint not a setting

PeterB: If you make an app that does things like echo cancelation and voice enhancements, they mess up things like music
... browser has no sane way to set sane defaults for music vs speach
... as more things get added to the browser, the music app will need to know to turn it off if we don't have something like this

Question about is auto value of enum or antoher flag.

Dan: We have an "auto" setting for constrainable properties. This would allow the browser to use the seperate hint about media type (music/speach) to do this

Adam: propose that is what we have today and this is what no constraint means

Dan: OK, perhaps don't need auto

Agrement in room we do not need to add Auto

Question about if put it souce / sink

JanIvar: lean towards source as it is descriptive of the track

Adam: we don't have controll of all the sinks so better not to it there. The fact that it might look like a camera, but it is a source. So want to put on a Track.

Proposal put it on track. (and be great if we could hear this from Randal if he is OK with this)

Questioon what happens if you have a track and you change the value of the hint.

Dan: as long as browser is operatinng with constraints, it can do what it wants and use info inculding this hint to decide how to process it

JanIva: Set echo cancel false. Then set hint to voice. This will not chang echo cancel as it was explicitly set. But if it had not been explicitly set, then this hint might chang the echo cance

Question about impact on GUM and extentiosn that don't know about it

Dan: Do we need to say anything about what extention specs must say? and other thing how does this change implemetation of existing specs.

Adam: we will have some hints, and if you exptend the base spec, you start with an empty bucket of items and so there is no impact of adding this

Dom: we should have words on if you are a sink you need to consdier the impact of hints
... do we try and bring that into GUM 1.0 or if this is a next version thing
... answer depends on who plans to implement

Bernard: one place this made a big differnces is when we do AV1 which has screen content encoding - prob bit longer than 6 months

Question how would conflict setting be addressed

Dan: There may be different places where you set the track. If you set it once and set it again, it replaces the previos. How much info carries to toher side. Questions on where you can set it.

JanIvar: any idea that this would cary across RTP over a clone.

Bernard: the decode knows what tools were used to encode but does not need to set the hint. Or the app might know from JS to turn it on. Would not assume it is carried over.

Dan: what about clone

DOm: wes should copy hint

JanIvar: this is a weak hint and you send it to web audio, hint gets dropped
... if there was no processing, the hint would be lost

<burn> ouch

We need to avoid slipper slope of speach/child

Dom: how to we deal with timeline of changes to attribute and when it happens to media

PeterT: no gaurentee of when it applies. It's a hint

What are audio category

speach and music seems to be the winners

Dom: so does speach and music corespond to what we need to adjust.
... is there a ref set of categories we could use from elsewhere to guide this

fluffy: existing codec have speach and music

Agrement in room: speach and music

Moving on to question to video categories

Bernard: even thought video came from screen, if it was a game, it might be motion not slides

<dom> https://wicg.github.io/mst-content-hint/#video-content-hints uses "motion" and "detail"

Adam: are we dupliating existing constraints ?

Fluffy: propose motion, animation, detail

<burn> (sorry about the echo - trying to find it)

Bernard: might effect the pallet used for encoding

Room is good with having at least motion detail

Question do we add third category of animation

Varun: will this confuse people
... will people know what detail is

Dom: we will have to provide good definitions

<stefanh> webrtc-pc talks about framerate/resolution https://w3c.github.io/webrtc-pc/#dom-rtcdegradationpreference

Dan: if input is a movie, don't change white balance or de nosie
... but if it is a camera would be good to do

Varun: what if person says want motion and detail

Dan: think abotu this as use case "this is what is going wrong", and thus "this is the hint you need"

<burn> varun asked what about motion and no-motion

<burn> then Peter said no-motion doesn't need a codec

Fluffy: think abotu example with comptuer vision and no codec

Adam: why do need the hint

Dom: hint is only usefull if there is something delegated to the browser

Adam: can we do this already

PeterT: for many things yes, we have direct control, but for some new things we don't have, having a hint helps

Varun: hints should not be on get user media

<dom> Accelerated Shape Detection in Images I mentioned earlier

JanIvar: the gum makes a track, and then things would allow setting it
... setting low framerate is not good way to say prefer high res

Varun: in screen capture of games, want high fps and good detail

JanIvar: track sits between soruce and sink. It is a way to configure source but it is also may have to become an API for sync too - for example downscale

Back to quesiton on do we add thrid animation category ?

Dan: What is the use case

Fluffy: arch fly through with synthetic artificial images

Varun: because it is articfical mostion, we want to see all of it in games and such

PeterT: had asked what you do because of this
... hint will not cover all use cases

Fluffy: would turn off pre processing of white balance, de-noise, etc. Would pass differnt paramters to AOM1 codec.

soares: want a way to turn off much of the pre and post processing

varun: want fine things or don't mess with motion

Adam: we are duplicating what might have in constraint

Fluffy: this is a "do what I mean" not "do what I said" interface

Dom: hard to know we need for do what I mean. Less convinced this model is good

<stefanh> What would the default be between motion and detail? "balanced"?

Dan: sugest we need more work on why this is needed

PeterT: two things going forward. 1) recovince ourselves we are OK with this model 2) do we want to add a third

JanIvar: like to see some examples of things we use this for that we don't want to do with direct controlls

Varun: people do not want to figure out the the constraints and instead just leave it empty and have the hint do the right thing

<dom> [we need a boolean "doTheRightThing" constraint]

PeterT: uefull to have more specific of how an browser might implement this and how apps might use it

<burn> [a Do What I Mean API is perfect, of course. If we had a Jarvis we could ask him to create that API]

Moving on to Issue 1533

Clarify whether RTCRtpContributingSource members are live. #1533

<dom> Clarify whether RTCRtpContributingSource members are live. #1533

JanIvar presenting: We could get live objects where you get an object and get live values of it.

This does not work becuse you don't get new particpats. And get old values for person that left.

Conclusion: live objects are not usefull

fluffy: why did we make it an interface

Adam: so do we have to poll

JanIvar: yes
... used to be one field, but now two. Why did we split

Dom: to make it cleaner ???

Agreed no live objects

Moving on to splitting priority up to two controlls

RTCPriorityType combines relative bitrate with QoS priority, which applications may not want. #1625

PeterT: made PR to deal with this

<dom> RTCPriorityType combines relative bitrate with QoS priority, which applications may not want. #1625

<dom> Let javascript set different priorities for bitrate and DSCP markings. #1659

dom: prefer clean break

JanIvar: is there an impact on data channel

Bernard: think it is OK

Fluffy: on behalf of HTA, do we have people that set it seperate

PeterT: Yes, we willhave people that set marking but not bitrate

<burn> Cullen: and we will need to set bitrate.

Varun: prefer ratio bitrates
... would use bitrate prioriyt with the 4 values

Dom: was trying to get to if the bitrate priories in 4 bin was worth doing now and would used. Is there enough value that we do this

Vaarun: fine with 8 4 2 1 locking

Summary: If the IETF is ok with peters changes, then WebRTC is in favour of making this changes

<adambe> scribenick: adambe

Media Capture, WebRTC-PC and WebRTC-Stats Next Steps

bernard: 15 issues relates to peer identity

fluffy: a lot of issues on peer identity are bogus
... no one is working on it

burn: Who's working on implementing (peer identity?)

bernard: We should look at interop requirements

fluffy: Our current text about two implementation is wrong
... it's more correct to say that there will be more than one browser

dom: Is there a difference?

bernard: It's also a question about how code is integrated in a browser

dom: The JS API is a small part of this
... What we want to verify is that it actually works to send media between browsers

burn: It's also important for independent people to be able to to implement the spec

bernard; Our edge-stack is fundamentally different

fluffy: It's hard to define independent implementations is our case; and it doesn't buy us much

dom: We could probably get away with only testing the JS API, but I think we have a moral duty to do more (i.e. interop on media level)

bernard: Do we need to change our text in the spec about "independent implementations"?

burn: We need to file an issue to update this text

AP on burn to file an issue

fluffy: It's in our charter that we should demonstrate media interop

bernard: Moving to other specs in the group
... We should have wider review of screen capture spec
... We need to check with the "depth" editors on the status
... The major action items are around screen capture

jib: Several browsers have some kind of implementation of screen captuer

<dom> Screen Capture editors draft

burn: The API should be standardized

fluffy: That's the easy part; the hard part is the privacy aspects

bernard: We should include "screen capture" spec in the interim meetings

AP on chairs: Bring screen capture in to the interm agendas

Topic is: WebRTC stats status

varun: summarizes the current issues

soares: It's quite straight forward to test the stats stuff
... We just need to put in the work

dom: Have you considered testing the life cycle of stats objects?
... We don't need to check the exact value of metrics. Just that they are not zero

varun: It's not easy to do
... That seems like browser testing to me

fluffy: It would be really nice to at some point test a bunch of stats between a set of browsers and verify that the result makes sense

varon: That kind of activity could be part of the "ultimate goal"

jib: Make sure we distinguish between API tests and "browser functionality tests"
... What about non-MTI stats?

varun: We're focusing on the MTI-ones at the moment
... Do we want allow experiments with stats?

fluffy: we can't disallow it
... Don't use prefixes on names

varun: Google has an old and a new API. Experiments are currently done in the old API

Discussion about experiments with origin trials

scribe: and different approaches on doing experiments with stats

<dom> ScribeNick: vr000m

WebRTC NV / New work

fine grained control for media, SVC, simulcast support.

Cullen: talking about Simulcast support.
...: we have level of simulcast support today, however, we do not have everything.

Peter Thatcher: for the server to negotiate simulcast, for the endpoint to receive simulcast (i.e., run the transcoding bridge on the browser)

Cullen: Fine grain media stack control, we could take a high level api is the pc spec, today. The ORTC stuff is moving away from SDP, but it still follows the O/A.
...: so we should expose all the minimal features to the javascript.

Pether Thatcher: ideas on WebRTC next steps

Cullen: can do codecs via webassembly should explore that

Alex: Explore the idea of using EME for hardware crypto

PeterT: QUIC recap
...: QUIC features for datachannels
... QuicTransport
... QuickStream discussion about ordered and unordered

Adam: there should be a way to get an event when data is acked on the send side

jib: did you use "async for" from whatwg streams

peter thatcher: there are some roadblocks.

<jib> "async for" == for-await-of https://github.com/tc39/proposal-async-iteration

DECISION: Add a constructor to the RTCIceTransport to the WebRTC CR

Peter can write other docs for Quic and other changes

<soareschen> Cullen: How to make ICE easier?

Wrap Up Items

<dom> ScribeNick: soareschen

Jan on issue 434

Disable user media by default in cross-origin iframes #434

<dom> Disable user media by default in cross-origin iframes #434

<dom> Feature Policy

Chrome has different syntax for iframes, i.e. allow="microsoft"

<dom> allow attribute

Jan: We should change the language in the spec for feature policy

Related issue: 440

Adam: If Chrome is implemented that way and mozilla is considering, we should revisit that

Dom: agreement
... Do mozilla expected to do both?

Jan: just the attribute

Dom: we can still move to PR if implementers already align with the changes
... reopen issue 440

Jan: does absence of allow='microphone' means disallowed?

Dom: we could just reuse the sandbox

Peter: more scary proposal

<dom> Dom: I'll make sure to follow up internally to see if feature policy can move to a more formal status

Audio-video RTC over QuicStream

way to get audio/video over any transport

Peter: low level way: stick in a track and get out a video frame
... for decoder, provide a frame, and at future time will go into a sink

Cullen: the frame need to include details such as timestamp

Does this mean the stream need to be ordered?

Peter: only needed for key frames

Dom: there are requests getting this feature into service workers

Vr00m & Peter: flexibility of inspecting metadata of encoded frames

Vr000m: Supporting SVC over QUIC?

Cullen: JS is fast enough to do SVC in JS

Vr000m: more metadata such as bitrate and target
... we can get a lot more control over this
... doing simulcast?

Peter: We could have 3 decoders
... We can get extra metadata such as layer for simulcast

Vr000m: Is it only one frame one frame?

Peter: You can have your own framing scheme for multiple frames in one stream

Cullen: When decoding one frame may be decoded into many frames

Peter: Haven't thought through for multiple frames

Dom: dictionary format for codec metadata
... are only codecs exposed are codecs that can be specified as dictionary?

Peter: there are two ways, not sure which one is better

Vr000m: on jitter buffer

Peter: you can have control with your own jitter buffer implementation

in JS

Vr000m: want to control jitter buffer underneath to send only one frame

Cullen: we can inspect timestamp metadata

Peter: we can specify frame dependency

Vr000m: if there is a jitter buffer underneath I want more control over it

Peter: example for audio encoder/decoder
... audio jitter buffer is different. Sink is pulling at regular rate regardless of whether it is filled
... if there is nothing you'll get silence

Vr000m: For video, we want to pass partial frames to video decoder

Peter: for media over QUIC streams, we can experiment more easily

without having to standardize first

Dan: did you talk about isolation?

Cullen: we can't have isolated media streams for this

Peter: one way is that the encoder can encrypt

Cullen: Or we can have isolated JS that does decoding

Jan: Audio worklet can run limited JS in audio thread
... Decoding video on main thread isn't desirable

Peter: technically we can implement encoder/decoder and jitter buffer in JS

only difference is you can't get access to native or hardware implementations

Cullen: You can build your own jitter buffer in JS, but most would use a default native implementation

Peter: option B: high level send/receive example
... require mapping down to QUIC stream

Dom: Youenne have some security concerns

Cullen: if isolation is a concern, can tell the encoder to pass only encrypted SRTP packet
... For decoder, pass SRTP packet to it which connect directly to sink
... People who use this API probably care less about isolation

<dom> Media Source Extensions

??: People might say we don't need another JS interface for audio/video pipeline

Peter: does WebAudio has encoders?


Scary ICE proposal

Cullen: Scary ICE proposal
... API to send STUN requests
... All the stuff stay the same. On the other hands we have API to nofify STUN requests

Things complicated for ICE is that we have two peers that test connection independently

Cullen: Move complexity of ICE to JS. no need to do it billaterally
... no need check list and gatherer state
... JS shouldn't able to modify STUN packet content
... browser would keep track of best one found
... but JS can decided which one to use
... we can find paths that ICE can't currently find

vr000m: Is it because current ICE have different priority?

Cullen: we don't want to allow JS to construct custom STUN packet to exploit the server

Peter: as long as there is limitation on how to send STUN packets it should be fine
... if we call send too fast, browser just return error

Cullen: browser always send STUN response to STUN packets

Peter: we also want notification responses

Cullen: problem of ICE is it only discover 2-way paths, not one way
... enterprises often have firewalls and want 1-way path

Peter: there are strict rules on how much/fast you can send in ICE, and we would still make those apply

Cullen: we are very careful to keep the security properties of ICE

Peter: we follow the ICE rules for punch etc

Cullen: I am even willing and able to write a test for this

Wrap up and actions

Bernard: wrap up

Cullen: for encoder/decover we prefer option A over B

Peter: we want to write an ICE transport doc

Adam and Dan: the testing PR #1657 has been merged, and there is now a testing.md with help

Bernard: do we have conclusion on SVC?

Cullen: we agree to not add new functionality for current spec

Dom: we should consider one document for each new feature
... we need new documents for QUIC stream and ICE transport extension
... encoder/decoder might have much more impact beyond WebRTC

Cullen: the proposal would be have a subset for the ICE protocol in IETF, then a W3c document for API access

Project snowflake for new ICE protocol

that's all for TPAC!

Summary of Action Items

Summary of Resolutions

  1. We adopt a test as you commit policy as describe in issue 1617 - we may revisit if it proves too complex in practice
  2. we don't change how offerToReceive*: false behaves in the spec
[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.152 (CVS log)
$Date: 2017/11/22 13:41:12 $