See also: IRC log
<inserted> scribenick: adambe
hta: the best use of face-to-face
time is to be specific
...: people pay more attention campared to mailing lists
... we've tried to make the agenda specific
... we might go with presenters proposal
... there also be discussions
... we should record off-topic topics and move them to the AOB session
stefanh: fist session is about
API functionality we haven't addressed yet
... this might be the least concrete item on the agenda
... what de we need to do now and what can we postpone
stefanh: API topics
... width, hegiht sent over a PeerConnection
... is media flowing?
... how much bandwith is used
... how much can be used
... how to reject media streams
... echo control
... one thing missing on the slides
... should we support security descriptions
(a use case is presented)
scribe: one peer sends video with
two different resolutions to two other peers
... are people ok with this?
fluffy: we should support this
burn: you're showing the entrence and the consumer side
hta: there are multiple ways to
achive this reuslt
... first question is.. do we need to support this?
... the answer is yes
... then how should we solve it
stefanh: this is the most
difficult slide in my deck
... application knows about the receiving side
... or the receiving side could tell the sender
... third possibilty is that the app isn't involved
matthew: even if the receiver is involved the sender needs to be involved
justin: there needs to be some higher limit
matthew: there might be a lot of contstraints
fluffy: this is one of the reasons we need constraints
stefanh: we shouldn't modify the sdp unless we have very good reasons
hta: we currently don't use a speakers queue
ekr: what do we expect to happen
if the receiver asks for a different aspect ratio
... should the PeerConnection rescale
justin: the sender might need to chose the closest thing
hta: we want people to separate the case where you really require something compared to the case where you really can't live without something
<matthew> side comment: generally we either need to fully specify the SDP the is used to negotiate things like this OR we need contraints. unfortunately when there's contraints, we *also* need to fully specify the SDP that flows between them... unless we choose to have it *only* be an API issue ("direct manipulation" vs. "constraint setting")
stefan: is the sending app or sending PeerConnection informe
justin: information about the consumer dimensions could be sent as a session description
fluffy: the connection to the
video tag is interesting
... would it be reasonable to adjust the send size to the consumer size?
... this was a driver for renegotiation callback
martin: you don't have the consumer in the initial negotiation
fluffy: it's not always true
derf: css also impacts this
fluffy: a video tag might be specified with a size or not
stefanh: there could be several video tags that wants to affect the sender side
hta: there's a ultimate fallback video size
burn: smart browsers might behave differently
matthew: smart browser decisions must be overidden if the developer wants to do something else
burn: if we have an API then the browser must take input from the API into account... otherwise the browser can do its magic
stefanh: what is the
... the receiving side can influence, but we also need an API to let the developer tweak
ekr: do we want information from the receiver to bubble back all the way to the camera on the sender side?
burn: a browser can use any valid value in a contraint range
matthew: if several consumers (local, remote).. who wins?
ekr: some constrains are enforced when set and some constrains seems to be enforced continiously
burn: we might need a constraints that says "do not change video size"
stefanh: recap on proposed
... receiving side as driver with an API for the app to influence
matthew: there's two ways to implement this
fluffy: we don't know if we'll use RTCP or SDP to negotiate video size
matthew: even if we go with RTCP we need an API to influence how the browser should behave
stefanh: let's move on
... next slide
... if the receiver detaches a stream from the consumer.. what should happen,
ekr: what if sender offers something that the receiver doesn't want?
stefanh: we'll get to that in a later slide
fluffy: we have two operations.. mute (undetectable from the sender side) and to say "don't send me packets"
justin: how do you say that I
only want a subset of the offered set of streams
... we don't have a good way to express that today
stefanh: next slide
... requeseting a certain bandwith
... there's an IETF draft talking about this
DanD: I propose a priority param
... we can discuss how the priority information should be handeled but we should decide on a mechanism to specify the intent
stefanh: addStream() is only called once per stream, it's a problem
DanD: we're the right group to specify the priority intent
burn: intent means it might not happen
<martin> hmm...s/constraint/pref/g ?
<martin> that works too
<matthew> there's a difference between "preferences" and "absolute limits"
<matthew> i "prefer" 720P, but there's no way i ever want >2560 horizontal pixels
<burn> no, constraint != intent
DanD: at least the browser sholud be able to say that I marked you packets in a certain way
<burn> intent == hint, discussions about hints lead to a decision to support both mandatory and optional constraints
<matthew> if you just go with direct manipulation then the API only has "settings", and the difference between "constraint" and "preference" is JS code
<matthew> but the moment you let the two ends talk to each other via possibly-unmodified SDP blobs then you need places on the monster where it can be poked with a stick
Göran: we should describe this case in more detail
scribe: can the network trust the diff-serv code points set by the app
fluffy: in some environment you
... they might provide information
... but a separate request is needed to verify
Göran: we need more discussion around use cases
fluffy: I think this should be a topic for the AoB section
hta: new topic - Security for Qos flow labels
ekr: there might be two API surfaces needed: tie constraints to media flow, (missed the other one)
DanD: I wan't a request that should be handeled by the trusted environment
matthew: even if diff-serv cps isn't supported a system may enforce priority anyhow (we need to support this case as well)
hta: new topic for AoB - Control of the DSCP interface
stefanh: next slide
... other side of the same coin
... feedback on bw..
martin: I thin we need generic
feedback on constraints set
... there might be existing mechanisms
... for others we might need to add something
matthew: e.g., if video has highter priority over slides, the app might want to know if the slides can't be sent at all
stefanh: next slide
... sender side pause/resume
... we currently have enable/disable on MediaStreamTrack
... every consumer is affected
... we could have enabel/disable on PeerConnection
burn: we don't need that since you could clone media streams
hta: we implemented enable and disable as affecting the associasion between a stream and track
stefanh: next topic, AGC
fluffy: the text on the slides looks good to me
<martin> conclusion here was that we don't need to prioritize the agc/noise settings, it's not a preference, it's a setting
<martin> it might also be necessary to change this on the fly
burn: what happens if you turn AGC on and it's not available
fluffy: it should be a on/off setting
derf: I hope there's a way to enumerate settings
<martin> we need a way to enumerate settings, for sure
fluffy: we should require that
<martin> agc/noise is a property of a track
hta: AGC should be on track level
stefanh: next topic
... rejecting streams
martin: we need to know if
multiple streams are offered
... we need basic methods on the session description
derf: example: you offer me 10k streams.. how do I reject
DanD: it's not only if your device can do something.. there's also a question if your service can support this
martin: we need to know what you get before you can say what you want to accept
hta: if we have a mechanisms for the receiver to turn off a stream at any point.. would it be sufficient to turn it off rather than not accepting it?
matthew: it might be a resource question
ekr: I interpret it as: you set the remote description and you see what streams are offered
stefanh: is the sending app
informed about a rejected streams
hta: if the use case for rejecting streams is to save resources then the sender needs to know
matthew: it's important how quickly can this be done
fluffy: can someone send a proposal to the list?
(I missed to scribe a lot of stuff while I was talking)
stefanh: next thing - AEC
fluffy: one param you need is what's going to the speakers
<martin> new issue that is related to this, and one that I will bring up later when we talk about this: SDP describes RTP sessions. The one m= section can have multiple tracks. How do we learn of these other than just waiting for arrival of packets on one of those?
<hta> adambe said: I sent a proposal to the list with inbound and outbound streams as different objects, which might make this discussion easier; missed having feedback on that.
<matthew> another issue that's related to the last slide and not yet decided (and different in different implementations) is what to offer/send before user consent on a sending device occurs. one approach is to say "recvonly" and change to "sendrecv" when the consent happens, the other is to say "sendrecv" and send muted audio + black video until consent is received. recvonly->sendrecv takes the extra round trip to enable.
(scribe is missing a lot of stuff here)
burn: why do we need API for this?
fluffy: you might want to turn it off
burn: could this be a browser control?
someone: music is a use case where you wan't to turn EC off
matthew: broadcast is also a use case where you want to turn AEC off
martin: if the browser and user wants differents things, the user setting should win
stefanh: final slide - SDES
... wait until RTCWeb has discussed this
stefanh: that was easy
... summary slide
martin: you could get two different tracks - one with AEC and one without
hta: new AoB (MC TF) to topic - Multiple open of camera/mic?
martin: AGC, NR should be fairly simple
burn: rather than doing this right now we should sumarize what we have decided the last hour and update the summary
hta: there are things that needs to be discussed and some that just can be edited in
Acoustic Echo Cancellation
stefanh: we're 15 minutes before schedule
<markus> coffee break
<hta> anyone remote actually here?
<markus> getting started, markus is the scribe
<markus> fluffy showing slides
issues: what SDP do createOffer and createAnswer create and how they can be manipulated before passing them back to the browser
matthew:some changes are ok from SDP syntax perspective but not for the browser to act upon
goals: clear definition of SDP and error handling rules
how are new ICE candidates added to the set SDP
Matthew: when I ask which SDP is in use is it the one I have set or can it be something slightly different?
constraints: can be used to enable common use cases, but do not solve what can be changed
burn:constraints can affect what SDP gets generated in the first place so it does not need to be modified anymore
fluffy: the place to define SDP use is in the JSEP draft - latest draft has a start
<AndyHutton> remote audio not working
matthew:state transitions need to be taken into account, can't call everything multiple times
<matthew> specifically, setRemoteDescription("answer") is restricted to only-call-once even though we are saying that enforcement of these transitions is up to the JS
what can be changed between create* and setLocal/Remote
use cases: remove codec, change bw limit etc.
justin: enable/disable bundle is one case
adam:rejecting audio, getting video - is that with munging SDP?
fluffy:no, if we have explicit API then don't use SDP mungling
matthew:have a detailed set of cases
fluffy:grouping of cases:
fluffy:don't do RTCP mux via settings, not SDP
matthew:ptime for codecs
matthew:want to fix it
matthew:will take my cases to the list
[several people agree most things being discussed should be done via other mechanisms than changing SDP]
burn: nervous about limiting too much what in SDP can be modified
fluffy:changing SDP on the way between browsers can be done flexibly, changing between create* and set* in the same browser more limited
justin:don't understand why there is big difference between different manipulation "loops"
fluffy:positive list of what can be changed is needed, not the list of what can't be
provide a list of changeable things that is *guaranteed* to work
matthew: errors need to be specific on what is wrong
[argument over what the current spec (JSEP?) says...]
burn: many kinds of modifications will be needed before sending SDP out
burn:what you send out does not match with what you set with setLocal?
<ekr> who was speaking?
matthew:jsep-02 has fixed the ice password issue
fluffy:need to have the use cases that motivate changes
justin:needs coming up in the future anyway...
how much and what state does createOffer create? devices with HW codecs?
<matthew> jsep-02 fixed the "createOffer is optional" language, though whether or not createOffer creates state is true or not is questionable
<ekr> matthew: though, there's clearly an implication of that b/c of the validity window of the offer.
<ekr> i.e. during the callback.
martin:set down the list of things that MUST be possible to change
burn:is everything else MUST reject?
people have different interpretations...
explicit list of things that MUST be changeable, explicit list of things that MUST not be changed and what falls in between the browser need to explicitly error report if it won't support it
<martin> there is a list of things that we MUST be able to change; there is a list of things that MUST NOT be able to change, which trigger an error; everything else the browser MUST either accept or reject with an error
<matthew> silent failure is incompatible with the assert on the 2nd slide. either the browser fires an error (invalidating the assert) or it is lying to you in order to make that assert.
[the above three comments (markus, martin, matthew) try to record the consensus in the meeting]
AndyHutton: Configuration/settings? how do they relate to the MUST, MUST NOT, ...
fluffy:start with the list of use cases to get the MUST-be-changeable list
[cullen taking notes...]
adam: we will have APIs for some of these things anyway
fluffy:if we have an API to control what createO/A gives, is that not enough, do we still need to change SDP for those "features"
justin, matthew: there may be cases where SDP change is still needed
fluffy: use cases are still not clear
burn:do not worry about MUST-list anymore because anything on the MUST NOT list can still work...
still sensing consensus on having three lists: 1. MUST be changeable, 2. MUST NOT be changeable, 3. (default) Browser MUST give an error if it does not support it
when can two different video flows use the same m-line
Proposal: all codec parameters are the same, "content-label" is the same, are in same MediaStream
(hta, fluffy, martin debate the details)
next: How does createOffer know to offer receiveOnly flow?
want to receive video but don't have video camera
justin has a proposal, mind to write it down here?
matthew: can you put "send" in SDP before getting user consent?
or do you first have to use receiveOnly and add sendReceive on a separate O/A
ekr: how do we correlate multiple offered video m-lines to the multiple video streams the answerer has
next topic: how does createOffer decide to offer a data channel?
should OfferToExchangeData constraint be added?
matthew: data is a great idea, but SCTP is horrible.
fluffy:take this to the IETF
tim:SCTP was decided in Feb
<ekr> matthew: how do you really feel about SCTP?
DanD:data is easy within a single app, but trapezoid between two apps is more difficult
martin:issues could come also if the other device running the "same" app has constraints
consensus: don't add this constraint
will be discussed tomorrow with a proposal
next: How long is SDP from createOffer/Answer valid?
matthew:90 seconds would be an ultimate timeout
use case: the SDP is sent to the server for modification
should it be valid beyond the duration of the callback function
ice candidates etc. time out in matter of 10s of seconds
hta: time-to-live for the session description?
matthew:can createOffer be called again after getting the modified SDP back from the server?
matthew:proposes that createAnswer is valid only for the duration of callback and no longer
consensus: it must be valid at least for the duration of the callback function
ACTION: ekr to take follow-up to the list
slides about rollback and error handling left for now
(now lunch break until 1:30)
<fluffy> Cullen is scribing
HTA is presenting what chrome is doing
TURN and opus are scheduled to be in M24 all going well
DTMF is waiting on this group
Anant has a demo of firefox
THe demo allows login, gives a list of users, then you can call one of the other users
Have getusermedia, have peerConnection,
expectation to not have behind a flag in firefox 19
currently in firefox 18 in nightly builds behind a flag
have a fairly complete version of the DataChannel
showed cool file sharing with drag and drop using DataChannel
THey have DTLS-SRTP
ICE but no TURN
prototype of Identity working
Doing desktop first, then working on mobile
VP8, opus, and G.711 as codecs
HTA: There was a test web even 2 days ago
dom: The idea of these is to get lots people to develop and contribute test cases
… presentation of theory of testing
… Event led to 404 test cases
Anant: exceptions vs error callbacks. exceptions when you can detect error without blocking the main thread. Covers only very simple errors. In all other cases, error callback. So as policy, favor error callbacks because they don't block the main thread.
Cullen: so lots of functions will do both?
EKR: there's no way to get around exceptions occasionally.
Tim: what do you do if you pass in something that is not a function as the error callback?
Cullen: there are things that you should never see in production code, and those that have to be caught at runtime. The former are programming errors. We're interested in input errors.
Anant: what goes in spec is a list of things that must be exceptions and what should be callbacks. This will be the result of consensus of browser implementations.
Justin: so programming errors will be exceptions, bad SDP from the other side will be an error callback.
Anant: exceptions will be only for development. They shouldn't occur in deployments at runtime. In exceptions should include name and message, which all platforms support We should standardize the names of exceptions, message can be platform-specific, thought should be human readable. stack and linenumber are useful where available. We will inherit from Error object to make SDPErrorObject.
ekr: we should create new attribute for SDP error, rather than overloading linenumber.
Anant: yes, having different names helps. We should standardize on the ones that need tobe machine readable. Human-readable ones can be platform-specific.
Dom: W3C policy is to re-use existing names as much as possible.
Anant: we will use same object for error callback and exception. So name and message must be present in object passed to error callback. As with exceptions, human-readable properties can be platform-specific. For SDP errors may want to create a new property. Error callbacks should be mandatory. In current spec, they're optional. That makes it easy to make sloppy errors. If they're mandatory, at least you get an exception if you forget to define the error callback. It should never be the case that there's an error and nothing happens.
... In CreateOffer, exceptions: Invalid_callback, invalid_constraints, invalid_state.
Dom: these can be webIDL type mismatch errors. We don't need to specify them separately.
Anant: we can move invalid state to a callback (for cases where app violates the state machine that we define.) So we won't need to define any exceptions for (most?) functions. setLocal and setRemote can have invalid_sdp as error in callback. Assume that the success callback in set remote isn't called until the description has been fully applied.
Cullen: if have setRemote with provisional SDP and then later will apply final SDP. Consider the case where are parsing SDP and acting on it as you go along. Do you have to roll back? It may be hard to do that.
ekr: setRemote shouldn't generate callback until it's complete and in correct state.
Justin: we need a separate error to signal case that media system is hopelessly busted.
Cullen: we need two kinds of errors. Implementation will know whether the situation ishopeless or not.
Harald: Invald SDP indicates that you have been able to rollback, so it's not fatal.
Cullen: when SDP fails a syntax check and no state has been changed, as opposed to case where what you thought was a camera turns out to be a mouse.
ekr: there are cases where the error is reversible, but you're still screwed.
Harald: let's put Anant's proposal into the spec, and work out some of these details later.
Anant: we can leave the decision on which errors are fatal to the UA. Further discussions of specific error cases on the list.
Dan: are error callbacks mandatory?
Dom: sloppy programmers can always put in a no-op error callback. On the other hand, if we make them mandatory, we will have to include them in our examples.
RESOLUTION: ERROR CALLBACKS ARE MANDATORY. Anant to update spec.
So decided that invalid_state will be a callback no exception
Decided that for SDP, will add a sdpLineNumber
decided all errors callbacks will not be optional
Justin: when you call setLocal that's when UA starts gathering ICE candidates. Get callbacks as each candidate is gathered, another when all are gathered. If call createOffer before any candidates are gathered, it will have just your local address.
Justin: if you want to do trickle, have to get offer first and then gather candidates. So must get getOffer callback before all candidates are gathered.
Cullen: what things can cause gathering to begin? What indicates how many candidates to gather? How do you know when you're done.
Matthew: it has to be a settable parameter whether createOffer callback can fire before you have enough candidates to produce valid SDP. I don't like to get a success callback when the offer is not valid yet.
Cullen: one model is you can decide to wait until SDP is complete. Or we can say that you get callback immediately, then get another when you have all the candidates.
Harald: must have ICE candidates to be valid.
Cullen: but local address suffices for validity.
Justin: in Jingle can send offer without any candidates and it's fine.
ekr: it won't work with SIP.
Adam: we should have matching examples and callflows, including trickle.
Stefan: the Chrome implementation seems to work, with and without trickle. So it should be our reference.
Justin: we discussed calling createOffer and setLocal early to start allocation, and then add streams later
Cullen: we don't have mechanism for declaring dummy streams. What if had new method: gather n candidates.
Justin: that's like setLocal with empty m=lines.
Cullen: calling app has pretty good idea of how many candidates it will need. At least an upper bound.
ekr: two proposals for pre-allocation: - a direct instruction to peerconnection, or some sort of dummy SDP via createOffer and setLocal.
Cullen: we have 6 issues to discuss, all at least as big as this.
Harald: I'd like to get an overview of all the questions. for this issue, we have two questions: how to know when SDP is sendable, and how to do pre-allocation.
Matthew: what happens with resource reservation? What happens when you do all this gathering and find out that the camera isn't available any more?
Cullen: next issue. How does receiving side find out what's in the offer, so that it can show the user: Alice is calling and she wants audio and video.
Matthew: when you call setRemote, you have no idea how many times onaddstream will be called.
Cullen: general model is a callback for each structure, and a final callback saying that you won't get any more. questions: when do you know that you've gotten all mediastreams? When do you know you have all the tracks? When do the callbacks fire with respect to the setRemote callback?
ekr: at what point do side effects take effect w.r.t the callback? I think it must be before.
Justin: when does peerconnection.remotestreams get populated? When you get a stream, the tracks should be filled in already.
Matthew: parse them into the array, and then call processingcomplete.
Justin: how about a callback saying the remotestreams value has changed? (It would be an array value, so this callback would be called only once.)
Adam: it's more convenient to get the new stream in the callback, rather than having to parse the array to see what's different.
Justin: but it's important to know when the changes are complete, rather than getting changes one at a time.
<inserted> ScribeNick: martin
juberti: [presenting on states]
martin: does back to gathering happen for moving from "relay" to "all"?
juberti: you probably already have all the necessary candidates
matthew: is there any way to remove candidates?
juberti: ice restart
ekr: can we have multiple components with one completed and the other one with no candidates
juberti: we are in checking until all components have resolved
fluffy: checking encompasses frozen
matthew: restart doesn't affect active flows
matthew: why not just describe each of the states as they relate to the underlying states
fluffy: this is part of what I prepared for the last call on this topic and we rejected that
hta: there is no harm in playing
audio while video is failing and restarting
... is there any difference betwen connected and completed?
juberti: middleboxes might require the updated offer that would be triggered from the completed transition
martin: is this application driven or not, can the browser add new candidates and contine?
juberti: restart is required by RFC 5245
fluffy: the application is going to need to be involved
juberti: if you get a new NIC (e.g. WiFi) you might just trickle that
hta: transitions to starting are tied to user actions
martin: how does the new WiFi candidate fit into this?
juberti: that would trigger a transition to connected
fluffy: there is an implication that disconnected might transition to failed, in the case where you were connected and you disconnect then something failed
juberti: proposes changing name of "starting" to "noo"
<hta> IceConnectState -> IceConnectionState
juberti: remove onicegatheringchange, and provide just onicecandidate to fire when the gathering state is changed
derf: propose to rename iceConnectState to iceConnectionState or something
* missed the name conclusion
ekr: is this same as the other state machine, just with the states merged?
<hta> martin, name conclusion was to use IceConnectedState rather than IceConnectState (I think - my ears are going)
matthew: set...(answer) can't be
... why isn't createOffer and createAnswer shown on the diagram?
juberti: once you set...(answer), you may have removed some critical state for the offer, which invalidates some of the answers