Web Real-Time Communications Working Group F2F Day 1

20 May 2014

See also: IRC log


HTA, StefanH, Adambe, AdamR, JIB, EKR, Martin, pthatcher, Juberti, shijun, bernard, Fluffy, burn, JimB, AlexGouaillard, Dom, TedHardie, AndrewH, Uwe, DanDruta, RichardBarnes, Suhas, DiniMartini, Jesup (remote), Kiran (remote)
MaryBarnes, RomanShpount, SeanTurner, Giri, Jonathan Lennox (remote)
hta, stefanh
stefanh, adambe


<trackbot> Date: 20 May 2014

<stefanh> Agenda and stuff: https://www.w3.org/2011/04/webrtc/wiki/May_19_-_May_21_2014

<dom> ScribeNick: stefanh

Approval of minutes of telco April 28th: approved

<dom> previous WebRTC teleconf minutes


Next topic: Doohickeys, Justin to present

<dom> Justin's slides

Justin: doohickeys aka RtpSenders/Receivers
...first: basics

<mt_> The objects *formally* known as Doohickeys
...first: Problem statement: need to tweak on a per track basis how they are sent over the net
... e.g. bitrate limits, direction,...
... They may be needed to be changed during operation, hence cannot be done at addTrack/Stream time
... Object model slide
... for stats we solved it with one model

<scribe> ...new object for RtpSending/Receiving to work on track level

UNKNOWN_SPEAKER: but a track can have several encodings (layered codec, simulcast) - may be out of scope for 1.0 though

Magnus: RTP -retrans may also genereate extra SSRC for a track

Justin: showing Solution Diagram slide
... How to obtain the objects (RtpSender/Receiver): event onaddtrack delivers Receiver, addTrack delivers Sender
... This model is simpler than addStream generating several Senders

Martin: Agree. We had this discussion in the China TPAC, addTrack better for several reasons.
... got a lot of support

Justin: change quite straightforward replace Stream for Track
... special case: addTrack. A track can be part of several Streams, needs to be signaled to preserve that model.
... several options, not signal for example, signal all Stream associations, single stream association.

Ted: Let's say video is added to an audio only session.Full offer answer needed.

Detailed discussion of streams vs. tracks vs. SDP O/A.

scribe: complex.

Martin: I think we need to address the everything case. Lot's of details to sort though.
... basic thing signed on to: signal MediaStream configuration.

Randell: Onnegneeded will fire if video is added to a session.

Adam R: On same page as Martin -> everything

scribe: ugly, but doable

Justin: is this in scope for us?
... maybe we don't need to signal all changes.

Cullen: If we deleted one track from one stream, signaling will happen. If we add the track to a new Stream -> move situation
... do we need to signal everything? No something can go out of band.

Martin: not the right way to hink about this.
... would surprise people

StefanH: support signal nothing.

Justin: Compromise solution.
... addign a track that is already added: error, it's already added.
... but adding a cloned track is OK

Ted: asking clarifying question about one track in several streams.
... optimizing transport not sending same track several times even though in several streams.
... the streams can contain different number of tracks.

Randell: have discussed this, worked on designs, but not concluded AFA can remember.

Justin: not sure this is affected by this.
... should work fine.

(Slide: addTrack proposal)

Magnus: transmission should be optimized.

AdamR: do you want to simplify API, SDP processing or what?

Justin: optimizing the mental model. Make it simpler.
... simplify signaling.

PeterT: Wants a simpler solution that handles the simple case,

AdamB: optimize for using the Track, but make it possible to polyfill

Justin: make it possible to signal synch state, but not signal all changes
... simplest model makes polyfill harder, my compromise handles that.

Cullen: msid in SDP tells for a track which streams it is in. What if msid is null?

Justin: generate on sending/receiving side - can't be null.

Cullen: SDP changes must always be communicated.

Justin: yes

Cullen and Justin seem to agree around onnegneeded etc.

Martin: We should keep the current model. Tracks can exist without streams. Remove onnegotiatinneeded event when someone plays with the grouping.
... would need to be onaddstream

(details discussed - scribe not keeping up)

Justin: onaddtrack could indicate sequence of streams it belongs to

Martin: no way to navigate from a track to find all streams it is part of.

Ted: In practice a track is usually part of one stream only.
... does the application know in advance, then it can work fine with Justin's proposal.
... the advanced cases can be handled over the top signaling.
... i.e. get the common case with current semantics, advanced cases different.
... but the app needs to know in advance.

Justin: streams is about signaling synchronization. audio and video should be in synch, screen share need not be

AdamR: We're talking about two things: streams where tracks are added locally, nothing should happen on PC.
... scribe losing track of discussion ...
... AdamR OK with Justins proposal

Ted: there are cases when you may want to move things out of synch context, but app is aware and can handle that out of band

Justin: we have to deal with this since we before could add things to the container (=stream).
... What should we optimize for? We can return details.

<mt_> For the record, I remain unconvinced, mostly because I think that the polyfill is trivial with c=0..n, but far more complicated than c=0..1

AdamB: Why is a new stream created if none was provided with addTrack?

Justin: To make app writing simpler.

AdamB: Solvable anyway.

Justin agree, but convenient.

Cullen: can you add a track to two streams, and have it be in the synch state of both streams?

Justin: no, not with this proposal.

Peter: You can accomplish that on the receive side.

Martin: yes, but the signaling is different.

Justin: this is a change.

Cullen: I'd describe this that a track can only be part of one stream over a peerconnection.

Justin: yes.
... discussion about synch context....

Cullen: everything in a peerconnection has a common time base, but not synched.
... discussion about synch....a very significant change....

Martin: what advantage do we get from the simplification?

Justin: I am OK with several streams being possible for a track
... at addTrack time.

Cullen: I have never heard a compelling use case for one track being in several streams. I don't really care.

Conclusion: going with the vector.

But can't change after addTrack (need to remove and add again).

Martin: default for the second parameter should be null.

<jesup> I forget the usecase that was used originally for sharing tracks. I remember there was one. I don't care that much.... I think it was to combine with different sets of tracks (audio, video). Note that the receiver could do some of these (modulo sync) by reworking the tracks into streams.

<jesup> I'm good with what we ended up with

Justin: onremovestream. Never clear when they should fire. Take away, the tracks just end at remote side.

AdamB: a stream never ends in the current spec.

Martin: what about RTCP-bye?

Ekr: have to listen to ended on a track? Seems Ok to me.

Justin: showing the actual API. Should not be surprising.
... but a vector should be added for the stream's part

Sense of the room?

harald asking questions. Who thinks we should change to this track model (instead of stream) for addXXX for PeerConnection is good?

Cullen thinks premature to ask.

scribe: I like the proposal, but premature to decide

Should we pursue (with right to abandon later if not good)

on not.

<jesup> Persue

<Erik> +1 to pursuing it

Almost all wanted to pursue, no one wanted to stop it.

Consensus in the room to pursue this.

Cullen: can a Receiver do removeTrack?

<jesup> We ;lost the feed from the room.....

<Ted_> Just a second

<jesup> back

Justin continuing with the contriversial stuff
....Transports: ICE state, DTLS certs
... proposal to add a .transport property per RTPSender/Receiver
... Transports 1.0 API shown

Ekr: for clarification - something about bundling and ICE and stuff....endorsing this

Kiran: codec preferenses question

Justin: we've not at that slide yet
... it comes later

Martin: likes the idea, some worries - solvable

Justin: do you agree with the properties that I propose are _in_ ?

Martin: the first probably OK, the second one not sure

Ted and Cullen: agree on second certificate comment

Cullen: I love this in every detail!

Ted: Don't like the part about the remote certs. Details about JS trust certs and stuff
... afraid someone will eventually have JS flush certs (or something....scribe losing it)
... not sure it is worth it

Harald: Opposite react to Ted: JS wants to know cert changes e.g. for stats reasons
... but what about ICE components?

Justin: let's ignore comp. for now

Ekr: seems fine to me.

Martin: can we take advantage of that we have someone knowing web crypto in the room?

Barnes: they are just objects
... ArrayBuffer or ArrayBufferView - detail to sort later.

<Ted_> He said "octets", not "octets"

Justin: next, encoding parameters
... codecs and parameters payload types... can be read out in RtpSender
... and we can make changes, three types: those that do not need negotiation - signaling not needed, those that need negotiation (pause); but can't change things that would be incosistent with SDP
... when changing, onnegotiationneeded might fire, then SDP O/A is needed to carry out the changes.
... a consequence: changes that req nego needs to be clearly described in SDP

AdamR: special case involving codecs, tones, switch Opus -> G.711 -> Opus. Allowed?

Justin: no

Martin: Is the intent of the API to express desires or expose the state?
... on reading out things that have been changed but not negotiated - what is read out?

Cullen: I can propose text.
... The values should represent the actual encoding, not the asked for (but not yet negotiated)
... changes coupled to setLocal in soem cases, to get answer back in other cases

<jesup> So, I have more parameters that should be here. See https://www.w3.org/Bugs/Public/show_bug.cgi?id=15861 for some examples

Cullen: we can set a set of easy to follow rules for this

Martin: some things happen immediately, some require some negotiation,

Suhas: bitrates within bounds of b=AS

Justin: need to look more into this.
... related to congestion control

Randell: Supportive if this - helps stuff I've proposed before
... frameRate, motionpreferred, bitrate and resolution issues, target video quality,....

Justin: resolution and framerate already on tracks.

Cullen: how do we decide what we do on tracks and what we do on doohickeys?

justin: depends on if it happens in raw media domain or encoded domain

Randell: scribe lost track
... quality parameters goes into the doohickey

AdamR: we should have a clean separation, model and view

Justin: agree
... we already have current and proposed SDPs

Martin: there is value in knowing what the browser is doing
... e.g. read out music detector state

Randell: talks for the use-case: an application wants to control if continue sending a track if the bitrate is too low. This could be the control surface for this.

Justin: auto pausing: set a minimum bitrate
... prioritization between tracks
... we do not have well defined SDP for simulcast
... showing the EncParams for 1.0 that he proposes

Ekr: simulcast: need to do JS controlled clone track etc....

Cullen: do we know how to add Simulcast SVC later, what do we do?

Justin: sequence of encoding parameters
... add a scaler for the different streams, def 1.0, add 0.5, 0.25 etc.

Cullen: scale makes me said - but we can figure that out
... how would FEC work?

Justin: parameter on encoding params

Cullen: would be much happier to have a sequence for SVC/Simulcast than the scale thing
... on the fly design discussion ongoing....
... let's poke on the lack of SDP support for simulcast. Is that really true?
... simulcast can be done pretty quickly if we want do. I want to.
... if simulcast was ready before this text should it be in?

Justin: as long as not stalling this doc
... moving on to discuss scheduling

Kiran: something about codec preferences
... related to offer answer negoneeded ....

AdamR: example coding parameters - current vs proposed

Bernard: you can already do simulcast: JS controlled. Losing efficiency, but doable.

Peter: How do I know what to set for SSRC?
... future question.

Ekr: I can live with JS simulcast
... worried about changing codecs
... conflicts can be a headache

Martin: you have not been very clear on teh model to use, related to negotiating the requested changes, what happens if you are half way through a nego?

Cullen: active and proposed SDP

Martin: this changes what you will propose the next time you propose something. How long does the browser keep the proposal? Until used.
... active and proposed state

DanD: What about priority? Not shown on the slides.

Justin: should be on the encoding params. Left it out because we need to decide what it means, prio 1 compared to prio 2

DanD: I think the idea was that it is more of a desire than a setting. Do you see it coming into 1.0?

Justin: no, but if someone willing to do the work it could get in.

Harald: Would be nice to have here
... LGTM, but more discussion needed.

Justin: would people be ok with a pull request for the Transport part?

Harald putting questions

Unclear result

scribe: is transport state ready for pull req?

Result: new proposal with two objects, one for tranpsort state, one for DTLS certs
... jusstin will write up

Coffee break till 3:55

<scribe> new scribe will be needed after coffee

Error handing

<dom> ScribeNick: adambe

<dom> EKR's slides

ekr: programming error gets exceptions, all other get error callbacks
... web idl does type checking
... I don't like when things fail silently
... if people think failing silently is good, lets have the discussion now
... the spec is written as: follow these steps
... at the end it says: if the PC is close, don't do anything
... this is confusing
... proposal: if the PC is close, don't do anything (for any method0
... at teh point where the next item is picked from teh queue, we should check for errors

(missed what Martin said)

<mt_> proposal is to have an InvalidStateError thrown when any action is enqueued; have the corresponding action fail (with an InvalidStateError) when dequeuing an action

juberti: what's the arguing from not letting everything in the queue complete

ekr: I would vote for close() to be queued

hta1: close should be synchronous

jesup: I agree with hta1
... apps are doing this already

ekr: do people think we could drop errors?

jesup: that would be problematic with e.g. promises

ekr: createOffer in the same event loop iteration as close will never have its callbakc fired

jib: close comes from the app

juberti: invalid state may not be the right error

mt_: close called a 2nd time always work

ekr: we should not have check for closed in every method description

fluffy: is this getting into the minutes

<mt_> ok: proposal is to change the definition of "enqueue a task" as above

<mt_> close is synchronous

<mt_> multiple calls to close succeed

<mt_> when closing, all outstanding actions are cancelled and their callbacks are fired with a "cancelled" error

ekr: addStream behavior may be overtaken by events
... multiple calls to addTrack with the same track should fire ResourceInUse
... should addTrack be queued?

juberti: that would mean that a lot of other stuff needs to be async as well

ekr: we're already changing a lot of stuf

<mt_> addTrack could be made asynchronous

<mt_> but at the same time, asynchronous calls suck to use

<hta> you can't emulate a synchronous call on top of an async one (unless you're prepared to ignore the results).

jib: the queue exist to protect our state

adam: it's no point to have sync and async stuff to interact with each other

jib: you need the result from createOffer() to continue

mt_: you don't need the output of createOffer to add a stream (track)

juberti: what ways can addStream() fail (except the steram already added)
... ?

ekr: you don't think it will it fail if you ask for too many encoders
... ?
... it's only addTrack() and close() sync atm

juberti: removeStream() is being replaced with removeTrack()

mt_: if you add a track twice it's simply gets added

juberti: I'm having second thoughts about setConfiguration() not needing to be async

adam: if addTrack() is sync the tracks won't show up on teh next line of code

mt_: addTrack() should add the track async if there's something in the queue, not otherwise

fluffy: I'm fine if implementations do what mt said, but I don't want mandate it

<mt_> my point here was that it should be ok to run these void functions (i.e., those without callbacks) synchronously

adambe: it's strange if things get queued sometimes but run sync otherwise

juberti: addTrack() returns an object now
... what state is that object in?

<fluffy> http://mozillamemes.tumblr.com/post/79874049694/it-worked-fine-yesterday

jib: we should use promises as a measuring stick

erk: send on a closed DataChannel should throw

ekr: try to create a DTMFSender with a video track -> InvalidParameter

<JonathanLennox> Great big picture of the number

ekr: send dtmf when canSendDTMF is false -> InvalidState
... insert DTMF with bogus values should generate InvalidParameter exception

<stefanh> WebIDL way to clamp DTMF errors

<dom> scribenick: stefanh

Unused errors - remove them

(mentioned in Ekr's slide)


defined, never used

proposal: spec that it is used in the error callbacks

the error should be redefined to inherit from the MediaStreamError (or whatever the name is now)

skiped slide

addIceCand bogus data

lot's of TBDs in the spec

error with line no not useful with ice cand - almost no linces

Ekr proposing us to define new errors

failure callbacks for createOffer/Answer

but can anything go wrong here? Only internal error?

(resource reservation happening on setLocal/Remote)

Cullen: resource reservation happens at createO, but released at callback.
... what should happen at createO if the ua knows it does not have teh resources?
... something needs to fail somewhere

Justin: setLocal can fail, seems better to only fire errors there
... if you can have createO fail, do something, have it succeed, that may talk for errors on createO
... but I think we can have all teh errors coming out of setLocal

hta: createO guarantees that the resources are avail. createO should be able to fail with "resources unavailable"

Justin: agree
... needs more discussion on the error data.

failure callbacks for setL/setR

can fail for many reasons

needs to add more errors, e.g. peeridentity error, invalidstateerror, invalidsdperror

adambe: what does invalidstateerror mean for setLocal with an answer?

ekr: can't go to that state

cullen: difference between not valid sdp and one that is valid but not compatible with the offer

ekr: ...more discussion on incompatible vs. invalid

suhas: enough with one error: SDP was wrong plus line number

hta: the diff is what was wrong -sdp generating code or state handling

justin: some recoverable, some not

ekr: incompatible sdp: no of m-lines does not match for example

adambe: reading some parts out of the spec to clarify

Jonathan: is this for debugging code or to help apps recovery

Ekr: in some cases the other side's code is broken

Spontaneous errors: some errors just happens - TURN errors, DTLS connection error, ....

not tied to any api call

use a .onerror or something RTCRuntmieError object

with an explannation attribute

What about state: how to distinguish fatal from non-fatal

state needs to change first

the list of runtime errors - only three so far, two and a half so far

Cullen: we need to carefully look through what we have on doohickeys and so on

code need to work in production not only demos

hta: turn and dtls should attach to doohickeys

internal errors could be reported on statechangeerror

justin: specific ice gathering error should be defined

cullen: we should add those

justin: I will add them to my pull request!!

Jonathan: initital failure different from subsequent failure

justin: need to think more about that

jonathan: obviously refresh may fail

cullen: people are keeping two connections active

justin: you want to know why your active connection went down

and we want gathering errors, ice disconnect errors and some more stuff

WebRTC schedule

<dom> HTA slides

hta: discuss "the plan"

slide 3: assumptions

stop adding stuff after doohickeys

<JonathanLennox> Can the room camera be repointed at Harald?

slide 4: resulting workplan

scribe: leading to W3C last call in end of September

after resolving bugs till mid September

what has the highest likelihood of causing delay?

Cullen: trickle ICE, FEC stuff, almost all docs we depend on are WG docs on the IETF side, good thing

hta: the w3c chairs would like the rtcweb chairs push more for getting the docs we wait for more visible and get action

Ted: we can make them visible, but not sure we get action. We need to escalate in teh IETF. That is a real risk.

Dom: W3C can reference anything for a last call. Refs must be stabel at CR/PR stage.
... must be very stable at Rec of course, but that comes later

Ted: always risk for requests for normative changes during IESG review

Sean: recent history show that this can be made to work

hta: word of warning, this is not the end. We will get a lot of input during last call, and we also need to build a test suite

DanB: last call is essentially the first public review

Cullen: I'd like to discuss how we deal with screen sharing.

hta: controversial, put off in a separate document

ekr: there is a proposal from MT and Cullen
... requested feature

stefanh: separate document?

Ekr: OK

hta: get someone do the respec stuff

cullen: how fast can that happen?

dom, hta: weeks or quicker - depends on the editor

hta: Let's go with this plan.

Thanks for today, see you tomorrow

<Erik> thanks for scribing stefan and adam

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.138 (CVS log)
$Date: 2014-05-27 12:23:32 $