See also: IRC log
<trackbot> Date: 20 May 2014
<stefanh> Agenda and stuff: https://www.w3.org/2011/04/webrtc/wiki/May_19_-_May_21_2014
<dom> ScribeNick: stefanh
Approval of minutes of telco April 28th: approved
<dom> previous WebRTC teleconf minutes
Next topic: Doohickeys, Justin to present
<dom> Justin's slides
Justin: doohickeys aka RtpSenders/Receivers
...first: basics
<mt_> The objects *formally* known as Doohickeys
...first: Problem statement: need to tweak on a per track basis how they are sent over the net
... e.g. bitrate limits, direction,...
... They may be needed to be changed during operation, hence cannot be done at addTrack/Stream time
... Object model slide
... for stats we solved it with one model
<scribe> ...new object for RtpSending/Receiving to work on track level
UNKNOWN_SPEAKER: but a track can have several encodings (layered codec, simulcast) - may be out of scope for 1.0 though
Magnus: RTP -retrans may also genereate extra SSRC for a track
Justin: showing Solution Diagram slide
... How to obtain the objects (RtpSender/Receiver): event onaddtrack delivers Receiver, addTrack delivers Sender
... This model is simpler than addStream generating several Senders
Martin: Agree. We had this discussion in the China TPAC, addTrack better for several reasons.
... got a lot of support
Justin: change quite straightforward replace Stream for Track
... special case: addTrack. A track can be part of several Streams, needs to be signaled to preserve that model.
... several options, not signal for example, signal all Stream associations, single stream association.
Ted: Let's say video is added to an audio only session.Full offer answer needed.
Detailed discussion of streams vs. tracks vs. SDP O/A.
scribe: complex.
Martin: I think we need to address the everything case. Lot's of details to sort though.
... basic thing signed on to: signal MediaStream configuration.
Randell: Onnegneeded will fire if video is added to a session.
Adam R: On same page as Martin -> everything
scribe: ugly, but doable
Justin: is this in scope for us?
... maybe we don't need to signal all changes.
Cullen: If we deleted one track from one stream, signaling will happen. If we add the track to a new Stream -> move situation
... do we need to signal everything? No something can go out of band.
Martin: not the right way to hink about this.
... would surprise people
StefanH: support signal nothing.
Justin: Compromise solution.
... addign a track that is already added: error, it's already added.
... but adding a cloned track is OK
Ted: asking clarifying question about one track in several streams.
... optimizing transport not sending same track several times even though in several streams.
... the streams can contain different number of tracks.
Randell: have discussed this, worked on designs, but not concluded AFA can remember.
Justin: not sure this is affected by this.
... should work fine.
(Slide: addTrack proposal)
Magnus: transmission should be optimized.
AdamR: do you want to simplify API, SDP processing or what?
Justin: optimizing the mental model. Make it simpler.
... simplify signaling.
PeterT: Wants a simpler solution that handles the simple case,
AdamB: optimize for using the Track, but make it possible to polyfill
Justin: make it possible to signal synch state, but not signal all changes
... simplest model makes polyfill harder, my compromise handles that.
Cullen: msid in SDP tells for a track which streams it is in. What if msid is null?
Justin: generate on sending/receiving side - can't be null.
Cullen: SDP changes must always be communicated.
Justin: yes
Cullen and Justin seem to agree around onnegneeded etc.
Martin: We should keep the current model. Tracks can exist without streams. Remove onnegotiatinneeded event when someone plays with the grouping.
... would need to be onaddstream
(details discussed - scribe not keeping up)
Justin: onaddtrack could indicate sequence of streams it belongs to
Martin: no way to navigate from a track to find all streams it is part of.
Ted: In practice a track is usually part of one stream only.
... does the application know in advance, then it can work fine with Justin's proposal.
... the advanced cases can be handled over the top signaling.
... i.e. get the common case with current semantics, advanced cases different.
... but the app needs to know in advance.
Justin: streams is about signaling synchronization. audio and video should be in synch, screen share need not be
AdamR: We're talking about two things: streams where tracks are added locally, nothing should happen on PC.
... scribe losing track of discussion ...
... AdamR OK with Justins proposal
Ted: there are cases when you may want to move things out of synch context, but app is aware and can handle that out of band
Justin: we have to deal with this since we before could add things to the container (=stream).
... What should we optimize for? We can return details.
<mt_> For the record, I remain unconvinced, mostly because I think that the polyfill is trivial with c=0..n, but far more complicated than c=0..1
AdamB: Why is a new stream created if none was provided with addTrack?
Justin: To make app writing simpler.
AdamB: Solvable anyway.
Justin agree, but convenient.
Cullen: can you add a track to two streams, and have it be in the synch state of both streams?
Justin: no, not with this proposal.
Peter: You can accomplish that on the receive side.
Martin: yes, but the signaling is different.
Justin: this is a change.
Cullen: I'd describe this that a track can only be part of one stream over a peerconnection.
Justin: yes.
... discussion about synch context....
Cullen: everything in a peerconnection has a common time base, but not synched.
... discussion about synch....a very significant change....
Martin: what advantage do we get from the simplification?
Justin: I am OK with several streams being possible for a track
... at addTrack time.
Cullen: I have never heard a compelling use case for one track being in several streams. I don't really care.
Conclusion: going with the vector.
But can't change after addTrack (need to remove and add again).
Martin: default for the second parameter should be null.
<jesup> I forget the usecase that was used originally for sharing tracks. I remember there was one. I don't care that much.... I think it was to combine with different sets of tracks (audio, video). Note that the receiver could do some of these (modulo sync) by reworking the tracks into streams.
<jesup> I'm good with what we ended up with
Justin: onremovestream. Never clear when they should fire. Take away, the tracks just end at remote side.
AdamB: a stream never ends in the current spec.
Martin: what about RTCP-bye?
Ekr: have to listen to ended on a track? Seems Ok to me.
Justin: showing the actual API. Should not be surprising.
... but a vector should be added for the stream's part
Sense of the room?
harald asking questions. Who thinks we should change to this track model (instead of stream) for addXXX for PeerConnection is good?
Cullen thinks premature to ask.
scribe: I like the proposal, but premature to decide
Should we pursue (with right to abandon later if not good)
on not.
<jesup> Persue
<Erik> +1 to pursuing it
Almost all wanted to pursue, no one wanted to stop it.
Consensus in the room to pursue this.
Cullen: can a Receiver do removeTrack?
<jesup> We ;lost the feed from the room.....
<Ted_> Just a second
<jesup> back
Justin continuing with the contriversial stuff
....Transports: ICE state, DTLS certs
... proposal to add a .transport property per RTPSender/Receiver
... Transports 1.0 API shown
Ekr: for clarification - something about bundling and ICE and stuff....endorsing this
Kiran: codec preferenses question
Justin: we've not at that slide yet
... it comes later
Martin: likes the idea, some worries - solvable
Justin: do you agree with the properties that I propose are _in_ ?
Martin: the first probably OK, the second one not sure
Ted and Cullen: agree on second certificate comment
Cullen: I love this in every detail!
Ted: Don't like the part about the remote certs. Details about JS trust certs and stuff
... afraid someone will eventually have JS flush certs (or something....scribe losing it)
... not sure it is worth it
Harald: Opposite react to Ted: JS wants to know cert changes e.g. for stats reasons
... but what about ICE components?
Justin: let's ignore comp. for now
Ekr: seems fine to me.
Martin: can we take advantage of that we have someone knowing web crypto in the room?
Barnes: they are just objects
... ArrayBuffer or ArrayBufferView - detail to sort later.
<Ted_> He said "octets", not "octets"
Justin: next, encoding parameters
... codecs and parameters payload types... can be read out in RtpSender
... and we can make changes, three types: those that do not need negotiation - signaling not needed, those that need negotiation (pause); but can't change things that would be incosistent with SDP
... when changing, onnegotiationneeded might fire, then SDP O/A is needed to carry out the changes.
... a consequence: changes that req nego needs to be clearly described in SDP
...
AdamR: special case involving codecs, tones, switch Opus -> G.711 -> Opus. Allowed?
Justin: no
Martin: Is the intent of the API to express desires or expose the state?
... on reading out things that have been changed but not negotiated - what is read out?
Cullen: I can propose text.
... The values should represent the actual encoding, not the asked for (but not yet negotiated)
... changes coupled to setLocal in soem cases, to get answer back in other cases
<jesup> So, I have more parameters that should be here. See https://www.w3.org/Bugs/Public/show_bug.cgi?id=15861 for some examples
Cullen: we can set a set of easy to follow rules for this
Martin: some things happen immediately, some require some negotiation,
Suhas: bitrates within bounds of b=AS
Justin: need to look more into this.
... related to congestion control
Randell: Supportive if this - helps stuff I've proposed before
... frameRate, motionpreferred, bitrate and resolution issues, target video quality,....
Justin: resolution and framerate already on tracks.
Cullen: how do we decide what we do on tracks and what we do on doohickeys?
justin: depends on if it happens in raw media domain or encoded domain
Randell: scribe lost track
... quality parameters goes into the doohickey
AdamR: we should have a clean separation, model and view
Justin: agree
... we already have current and proposed SDPs
Martin: there is value in knowing what the browser is doing
... e.g. read out music detector state
Randell: talks for the use-case: an application wants to control if continue sending a track if the bitrate is too low. This could be the control surface for this.
Justin: auto pausing: set a minimum bitrate
... prioritization between tracks
... we do not have well defined SDP for simulcast
... showing the EncParams for 1.0 that he proposes
Ekr: simulcast: need to do JS controlled clone track etc....
Cullen: do we know how to add Simulcast SVC later, what do we do?
Justin: sequence of encoding parameters
... add a scaler for the different streams, def 1.0, add 0.5, 0.25 etc.
Cullen: scale makes me said - but we can figure that out
... how would FEC work?
Justin: parameter on encoding params
Cullen: would be much happier to have a sequence for SVC/Simulcast than the scale thing
... on the fly design discussion ongoing....
... let's poke on the lack of SDP support for simulcast. Is that really true?
... simulcast can be done pretty quickly if we want do. I want to.
... if simulcast was ready before this text should it be in?
Justin: as long as not stalling this doc
... moving on to discuss scheduling
Kiran: something about codec preferences
... related to offer answer negoneeded ....
AdamR: example coding parameters - current vs proposed
Bernard: you can already do simulcast: JS controlled. Losing efficiency, but doable.
Peter: How do I know what to set for SSRC?
... future question.
Ekr: I can live with JS simulcast
... worried about changing codecs
... conflicts can be a headache
Martin: you have not been very clear on teh model to use, related to negotiating the requested changes, what happens if you are half way through a nego?
Cullen: active and proposed SDP
Martin: this changes what you will propose the next time you propose something. How long does the browser keep the proposal? Until used.
... active and proposed state
DanD: What about priority? Not shown on the slides.
Justin: should be on the encoding params. Left it out because we need to decide what it means, prio 1 compared to prio 2
DanD: I think the idea was that it is more of a desire than a setting. Do you see it coming into 1.0?
Justin: no, but if someone willing to do the work it could get in.
Harald: Would be nice to have here
... LGTM, but more discussion needed.
Justin: would people be ok with a pull request for the Transport part?
Harald putting questions
Unclear result
scribe: is transport state ready for pull req?
Result: new proposal with two objects, one for tranpsort state, one for DTLS certs
... jusstin will write up
Coffee break till 3:55
<scribe> new scribe will be needed after coffee
<dom> ScribeNick: adambe
<dom> EKR's slides
ekr: programming error gets exceptions, all other get error callbacks
... web idl does type checking
... I don't like when things fail silently
... if people think failing silently is good, lets have the discussion now
... the spec is written as: follow these steps
... at the end it says: if the PC is close, don't do anything
... this is confusing
... proposal: if the PC is close, don't do anything (for any method0
... at teh point where the next item is picked from teh queue, we should check for errors
(missed what Martin said)
<mt_> proposal is to have an InvalidStateError thrown when any action is enqueued; have the corresponding action fail (with an InvalidStateError) when dequeuing an action
juberti: what's the arguing from not letting everything in the queue complete
ekr: I would vote for close() to be queued
hta1: close should be synchronous
jesup: I agree with hta1
... apps are doing this already
ekr: do people think we could drop errors?
jesup: that would be problematic with e.g. promises
ekr: createOffer in the same event loop iteration as close will never have its callbakc fired
jib: close comes from the app
juberti: invalid state may not be the right error
mt_: close called a 2nd time always work
ekr: we should not have check for closed in every method description
fluffy: is this getting into the minutes
<mt_> ok: proposal is to change the definition of "enqueue a task" as above
<mt_> close is synchronous
<mt_> multiple calls to close succeed
<mt_> when closing, all outstanding actions are cancelled and their callbacks are fired with a "cancelled" error
ekr: addStream behavior may be overtaken by events
... multiple calls to addTrack with the same track should fire ResourceInUse
... should addTrack be queued?
juberti: that would mean that a lot of other stuff needs to be async as well
ekr: we're already changing a lot of stuf
<mt_> addTrack could be made asynchronous
<mt_> but at the same time, asynchronous calls suck to use
<hta> you can't emulate a synchronous call on top of an async one (unless you're prepared to ignore the results).
jib: the queue exist to protect our state
adam: it's no point to have sync and async stuff to interact with each other
jib: you need the result from createOffer() to continue
mt_: you don't need the output of createOffer to add a stream (track)
juberti: what ways can addStream() fail (except the steram already added)
... ?
ekr: you don't think it will it fail if you ask for too many encoders
... ?
... it's only addTrack() and close() sync atm
juberti: removeStream() is being replaced with removeTrack()
mt_: if you add a track twice it's simply gets added
juberti: I'm having second thoughts about setConfiguration() not needing to be async
adam: if addTrack() is sync the tracks won't show up on teh next line of code
mt_: addTrack() should add the track async if there's something in the queue, not otherwise
fluffy: I'm fine if implementations do what mt said, but I don't want mandate it
<mt_> my point here was that it should be ok to run these void functions (i.e., those without callbacks) synchronously
adambe: it's strange if things get queued sometimes but run sync otherwise
juberti: addTrack() returns an object now
... what state is that object in?
<fluffy> http://mozillamemes.tumblr.com/post/79874049694/it-worked-fine-yesterday
jib: we should use promises as a measuring stick
erk: send on a closed DataChannel should throw
ekr: try to create a DTMFSender with a video track -> InvalidParameter
<JonathanLennox> Great big picture of the number
ekr: send dtmf when canSendDTMF is false -> InvalidState
... insert DTMF with bogus values should generate InvalidParameter exception
<stefanh> WebIDL way to clamp DTMF errors
<dom> scribenick: stefanh
Unused errors - remove them
(mentioned in Ekr's slide)
RTCSdpError:
defined, never used
proposal: spec that it is used in the error callbacks
the error should be redefined to inherit from the MediaStreamError (or whatever the name is now)
skiped slide
addIceCand bogus data
lot's of TBDs in the spec
error with line no not useful with ice cand - almost no linces
Ekr proposing us to define new errors
failure callbacks for createOffer/Answer
but can anything go wrong here? Only internal error?
(resource reservation happening on setLocal/Remote)
Cullen: resource reservation happens at createO, but released at callback.
... what should happen at createO if the ua knows it does not have teh resources?
... something needs to fail somewhere
Justin: setLocal can fail, seems better to only fire errors there
... if you can have createO fail, do something, have it succeed, that may talk for errors on createO
... but I think we can have all teh errors coming out of setLocal
hta: createO guarantees that the resources are avail. createO should be able to fail with "resources unavailable"
Justin: agree
... needs more discussion on the error data.
failure callbacks for setL/setR
can fail for many reasons
needs to add more errors, e.g. peeridentity error, invalidstateerror, invalidsdperror
adambe: what does invalidstateerror mean for setLocal with an answer?
ekr: can't go to that state
cullen: difference between not valid sdp and one that is valid but not compatible with the offer
ekr: ...more discussion on incompatible vs. invalid
suhas: enough with one error: SDP was wrong plus line number
hta: the diff is what was wrong -sdp generating code or state handling
justin: some recoverable, some not
ekr: incompatible sdp: no of m-lines does not match for example
adambe: reading some parts out of the spec to clarify
Jonathan: is this for debugging code or to help apps recovery
Ekr: in some cases the other side's code is broken
Spontaneous errors: some errors just happens - TURN errors, DTLS connection error, ....
not tied to any api call
use a .onerror or something RTCRuntmieError object
with an explannation attribute
What about state: how to distinguish fatal from non-fatal
state needs to change first
the list of runtime errors - only three so far, two and a half so far
Cullen: we need to carefully look through what we have on doohickeys and so on
code need to work in production not only demos
hta: turn and dtls should attach to doohickeys
internal errors could be reported on statechangeerror
justin: specific ice gathering error should be defined
cullen: we should add those
justin: I will add them to my pull request!!
Jonathan: initital failure different from subsequent failure
justin: need to think more about that
jonathan: obviously refresh may fail
cullen: people are keeping two connections active
justin: you want to know why your active connection went down
and we want gathering errors, ice disconnect errors and some more stuff
<dom> HTA slides
hta: discuss "the plan"
slide 3: assumptions
stop adding stuff after doohickeys
<JonathanLennox> Can the room camera be repointed at Harald?
slide 4: resulting workplan
scribe: leading to W3C last call in end of September
after resolving bugs till mid September
what has the highest likelihood of causing delay?
Cullen: trickle ICE, FEC stuff, almost all docs we depend on are WG docs on the IETF side, good thing
hta: the w3c chairs would like the rtcweb chairs push more for getting the docs we wait for more visible and get action
Ted: we can make them visible, but not sure we get action. We need to escalate in teh IETF. That is a real risk.
Dom: W3C can reference anything for a last call. Refs must be stabel at CR/PR stage.
... must be very stable at Rec of course, but that comes later
Ted: always risk for requests for normative changes during IESG review
Sean: recent history show that this can be made to work
hta: word of warning, this is not the end. We will get a lot of input during last call, and we also need to build a test suite
DanB: last call is essentially the first public review
Cullen: I'd like to discuss how we deal with screen sharing.
hta: controversial, put off in a separate document
ekr: there is a proposal from MT and Cullen
... requested feature
stefanh: separate document?
Ekr: OK
hta: get someone do the respec stuff
cullen: how fast can that happen?
dom, hta: weeks or quicker - depends on the editor
hta: Let's go with this plan.
Thanks for today, see you tomorrow
<Erik> thanks for scribing stefan and adam