- From: Stefan Hakansson LK <stefan.lk.hakansson@ericsson.com>
- Date: Thu, 15 Nov 2012 15:18:17 +0100
- To: public-webrtc@w3.org
Dom,
thanks!
I have only one comment:
* I think "[NEW] ACTION: Cullen to update draft to remove reference."
should not be here, from the context of the discussion I interpret that
it is an IETF document that should be updated.
Stefan
On 11/15/2012 11:37 AM, Dominique Hazael-Massieux wrote:
> Hi,
>
> I've just finished compiling and cleaning up the draft minutes of our
> F2F meeting two weeks ago at TPAC:
> http://www.w3.org/2012/10/29-webrtc-minutes.html
> http://www.w3.org/2012/10/30-webrtc-minutes.html
>
> The minutes of the Media Capture Task Force meeting on the afternoon of
> October 30th will be send separately to the media capture task force
> mailing list.
>
> Please send corrections as needed; text-only copy of the minutes
> embedded below.
>
> Dom
>
> Web Real-Time Communications Working Group F2F
>
> 29 Oct 2012
>
> See also: [2]IRC log
>
> [2] http://www.w3.org/2012/10/29-webrtc-irc
>
> Attendees
>
> Present
> andy_hutton, adambe, hta, stefanh, derf, burn,
> dan_druta, richt, anant, dom, juberti, jim, matthew,
> ekr, fluffy, Magnus
>
> Regrets
> Chair
> Harald, Stefan
>
> Scribe
> adambe, markus, JimB, martin
>
> Contents
>
> * [3]Topics
> 1. [4]API functionalities missing in PeerConnection API
> 2. [5]SDP handling
> 3. [6]Implementation status
> 4. [7]General error handling principles
> 5. [8]Call flows Walk-through
> 6. [9]State machines
> * [10]Summary of Action Items
> __________________________________________________________
>
> <inserted> scribenick: adambe
>
> hta: the best use of face-to-face time is to be specific
> ...: people pay more attention campared to mailing lists
> ... we've tried to make the agenda specific
> ... we might go with presenters proposal
> ... there also be discussions
> ... we should record off-topic topics and move them to the AOB
> session
>
> stefanh: fist session is about API functionality we haven't
> addressed yet
> ... this might be the least concrete item on the agenda
> ... what de we need to do now and what can we postpone
>
> API functionalities missing in PeerConnection API
>
> [11]Stefan's slides
>
> [11] http://www.w3.org/2011/04/webrtc/wiki/images/1/1e/MoreAPIs.pdf
>
> stefanh: API topics
> ... width, hegiht sent over a PeerConnection
> ... priority
> ... is media flowing?
> ... how much bandwith is used
> ... how much can be used
> ... how to reject media streams
> ... echo control
> ... one thing missing on the slides
> ... should we support security descriptions
>
> (a use case is presented)
>
> scribe: one peer sends video with two different resolutions to
> two other peers
> ... are people ok with this?
>
> fluffy: we should support this
>
> burn: you're showing the entrence and the consumer side
>
> stefanh: yes
>
> hta: there are multiple ways to achive this reuslt
> ... first question is.. do we need to support this?
> ... the answer is yes
> ... then how should we solve it
>
> stefanh: this is the most difficult slide in my deck
> ... application knows about the receiving side
> ... or the receiving side could tell the sender
> ... third possibilty is that the app isn't involved
>
> matthew: even if the receiver is involved the sender needs to
> be involved
>
> justin: there needs to be some higher limit
>
> matthew: there might be a lot of contstraints
>
> fluffy: this is one of the reasons we need constraints
>
> stefanh: we shouldn't modify the sdp unless we have very good
> reasons
>
> hta: we currently don't use a speakers queue
>
> ekr: what do we expect to happen if the receiver asks for a
> different aspect ratio
> ... should the PeerConnection rescale
> ... ?
>
> justin: the sender might need to chose the closest thing
>
> hta: we want people to separate the case where you really
> require something compared to the case where you really can't
> live without something
>
> <matthew> side comment: generally we either need to fully
> specify the SDP the is used to negotiate things like this OR we
> need contraints. unfortunately when there's contraints, we
> *also* need to fully specify the SDP that flows between them...
> unless we choose to have it *only* be an API issue ("direct
> manipulation" vs. "constraint setting")
>
> stefan: is the sending app or sending PeerConnection informe
>
> justin: information about the consumer dimensions could be sent
> as a session description
>
> fluffy: the connection to the video tag is interesting
> ... would it be reasonable to adjust the send size to the
> consumer size?
> ... this was a driver for renegotiation callback
>
> martin: you don't have the consumer in the initial negotiation
>
> fluffy: it's not always true
>
> derf: css also impacts this
>
> fluffy: a video tag might be specified with a size or not
>
> stefanh: there could be several video tags that wants to affect
> the sender side
>
> hta: there's a ultimate fallback video size
>
> burn: smart browsers might behave differently
>
> matthew: smart browser decisions must be overidden if the
> developer wants to do something else
>
> burn: if we have an API then the browser must take input from
> the API into account... otherwise the browser can do its magic
>
> stefanh: what is the conclusion?
> ... the receiving side can influence, but we also need an API
> to let the developer tweak
>
> ekr: do we want information from the receiver to bubble back
> all the way to the camera on the sender side?
>
> burn: a browser can use any valid value in a contraint range
>
> matthew: if several consumers (local, remote).. who wins?
>
> ekr: some constrains are enforced when set and some constrains
> seems to be enforced continiously
>
> burn: we might need a constraints that says "do not change
> video size"
>
> stefanh: recap on proposed conclusion
> ... receiving side as driver with an API for the app to
> influence
>
> matthew: there's two ways to implement this
>
> fluffy: we don't know if we'll use RTCP or SDP to negotiate
> video size
>
> matthew: even if we go with RTCP we need an API to influence
> how the browser should behave
>
> stefanh: let's move on
> ... next slide
> ... if the receiver detaches a stream from the consumer.. what
> should happen,
> ... ?
>
> ekr: what if sender offers something that the receiver doesn't
> want?
>
> stefanh: we'll get to that in a later slide
>
> fluffy: we have two operations.. mute (undetectable from the
> sender side) and to say "don't send me packets"
>
> justin: how do you say that I only want a subset of the offered
> set of streams
> ... we don't have a good way to express that today
>
> stefanh: next slide
> ... requeseting a certain bandwith
> ... there's an IETF draft talking about this
>
> DanD: I propose a priority param to addStream()
> ... we can discuss how the priority information should be
> handeled but we should decide on a mechanism to specify the
> intent
>
> stefanh: addStream() is only called once per stream, it's a
> problem
>
> DanD: we're the right group to specify the priority intent
>
> burn: intent means it might not happen
>
> <martin> hmm...s/constraint/pref/g ?
>
> <ekr> ?
>
> <martin> that works too
>
> <matthew> there's a difference between "preferences" and
> "absolute limits"
>
> <matthew> i "prefer" 720P, but there's no way i ever want >2560
> horizontal pixels
>
> <burn> no, constraint != intent
>
> DanD: at least the browser sholud be able to say that I marked
> you packets in a certain way
>
> <burn> intent == hint, discussions about hints lead to a
> decision to support both mandatory and optional constraints
>
> <matthew> if you just go with direct manipulation then the API
> only has "settings", and the difference between "constraint"
> and "preference" is JS code
>
> <matthew> but the moment you let the two ends talk to each
> other via possibly-unmodified SDP blobs then you need places on
> the monster where it can be poked with a stick
>
> Göran: we should describe this case in more detail
>
> scribe: can the network trust the diff-serv code points set by
> the app
>
> fluffy: in some environment you can't
> ... they might provide information
> ... but a separate request is needed to verify
>
> Göran: we need more discussion around use cases
>
> fluffy: I think this should be a topic for the AoB section
>
> (people agree)
>
> hta: new topic - Security for Qos flow labels
>
> ekr: there might be two API surfaces needed: tie constraints to
> media flow, (missed the other one)
>
> DanD: I wan't a request that should be handeled by the trusted
> environment
>
> matthew: even if diff-serv cps isn't supported a system may
> enforce priority anyhow (we need to support this case as well)
>
> hta: new topic for AoB - Control of the DSCP interface
>
> stefanh: next slide
> ... other side of the same coin
> ... feedback on bw..
>
> martin: I thin we need generic feedback on constraints set
> ... there might be existing mechanisms
> ... for others we might need to add something
>
> matthew: e.g., if video has highter priority over slides, the
> app might want to know if the slides can't be sent at all
>
> stefanh: next slide
> ... sender side pause/resume
> ... we currently have enable/disable on MediaStreamTrack
> ... every consumer is affected
> ... we could have enabel/disable on PeerConnection
>
> burn: we don't need that since you could clone media streams
>
> hta: we implemented enable and disable as affecting the
> associasion between a stream and track
>
> stefanh: next topic, AGC
>
> fluffy: the text on the slides looks good to me
>
> <martin> conclusion here was that we don't need to prioritize
> the agc/noise settings, it's not a preference, it's a setting
>
> <martin> it might also be necessary to change this on the fly
>
> burn: what happens if you turn AGC on and it's not available
>
> fluffy: it should be a on/off setting
>
> derf: I hope there's a way to enumerate settings
>
> <martin> we need a way to enumerate settings, for sure
>
> fluffy: we should require that
>
> <martin> agc/noise is a property of a track
>
> hta: AGC should be on track level
>
> stefanh: next topic
> ... rejecting streams
>
> martin: we need to know if multiple streams are offered
> ... we need basic methods on the session description
>
> derf: example: you offer me 10k streams.. how do I reject
>
> DanD: it's not only if your device can do something.. there's
> also a question if your service can support this
>
> martin: we need to know what you get before you can say what
> you want to accept
>
> hta: if we have a mechanisms for the receiver to turn off a
> stream at any point.. would it be sufficient to turn it off
> rather than not accepting it?
>
> matthew: it might be a resource question
>
> ekr: I interpret it as: you set the remote description and you
> see what streams are offered
>
> stefanh: is the sending app informed about a rejected streams
> ... ?
>
> hta: if the use case for rejecting streams is to save resources
> then the sender needs to know
>
> matthew: it's important how quickly can this be done
>
> fluffy: can someone send a proposal to the list?
>
> (I missed to scribe a lot of stuff while I was talking)
>
> stefanh: next thing - AEC
>
> fluffy: one param you need is what's going to the speakers
>
> <martin> new issue that is related to this, and one that I will
> bring up later when we talk about this: SDP describes RTP
> sessions. The one m= section can have multiple tracks. How do
> we learn of these other than just waiting for arrival of
> packets on one of those?
>
> <hta> adambe said: I sent a proposal to the list with inbound
> and outbound streams as different objects, which might make
> this discussion easier; missed having feedback on that.
>
> <matthew> another issue that's related to the last slide and
> not yet decided (and different in different implementations) is
> what to offer/send before user consent on a sending device
> occurs. one approach is to say "recvonly" and change to
> "sendrecv" when the consent happens, the other is to say
> "sendrecv" and send muted audio + black video until consent is
> received. recvonly->sendrecv takes the extra round trip to
> enable.
>
> (scribe is missing a lot of stuff here)
>
> burn: why do we need API for this?
>
> fluffy: you might want to turn it off
>
> burn: could this be a browser control?
>
> someone: music is a use case where you wan't to turn EC off
>
> matthew: broadcast is also a use case where you want to turn
> AEC off
>
> martin: if the browser and user wants differents things, the
> user setting should win
>
> stefanh: final slide - SDES
> ... wait until RTCWeb has discussed this
>
> ekr: yes
>
> stefanh: that was easy
> ... summary slide
>
> martin: you could get two different tracks - one with AEC and
> one without
>
> hta: new AoB (MC TF) to topic - Multiple open of camera/mic?
>
> martin: AGC, NR should be fairly simple
>
> burn: rather than doing this right now we should sumarize what
> we have decided the last hour and update the summary
>
> hta: there are things that needs to be discussed and some that
> just can be edited in
>
> Acoustic Echo Cancellation
>
> stefanh: we're 15 minutes before schedule
>
> <markus> coffee break
>
> <hta> anyone remote actually here?
>
> SDP handling
>
> [12]Cullen's slides on SDP handling
>
> [12]
> http://www.w3.org/2011/04/webrtc/wiki/images/2/2d/RTCWeb-SDP-API_v6.pdf
>
> <markus> getting started, markus is the scribe
>
> <markus> fluffy showing slides
>
> issues: what SDP do createOffer and createAnswer create and how
> they can be manipulated before passing them back to the browser
>
> matthew:some changes are ok from SDP syntax perspective but not
> for the browser to act upon
>
> goals: clear definition of SDP and error handling rules
>
> how are new ICE candidates added to the set SDP
>
> Matthew: when I ask which SDP is in use is it the one I have
> set or can it be something slightly different?
>
> constraints: can be used to enable common use cases, but do not
> solve what can be changed
>
> burn:constraints can affect what SDP gets generated in the
> first place so it does not need to be modified anymore
>
> fluffy: the place to define SDP use is in the JSEP draft -
> latest draft has a start
>
> <AndyHutton> remote audio not working
>
> matthew:state transitions need to be taken into account, can't
> call everything multiple times
>
> <matthew> specifically, setRemoteDescription("answer") is
> restricted to only-call-once even though we are saying that
> enforcement of these transitions is up to the JS
>
> what can be changed between create* and setLocal/Remote
>
> use cases: remove codec, change bw limit etc.
>
> justin: enable/disable bundle is one case
>
> adam:rejecting audio, getting video - is that with munging SDP?
>
> fluffy:no, if we have explicit API then don't use SDP mungling
>
> matthew:have a detailed set of cases
>
> fluffy:grouping of cases:
>
> fluffy:don't do RTCP mux via settings, not SDP
>
> matthew:ptime for codecs
>
> matthew:want to fix it
>
> matthew:will take my cases to the list
>
> [several people agree most things being discussed should be
> done via other mechanisms than changing SDP]
>
> burn: nervous about limiting too much what in SDP can be
> modified
>
> fluffy:changing SDP on the way between browsers can be done
> flexibly, changing between create* and set* in the same browser
> more limited
>
> justin:don't understand why there is big difference between
> different manipulation "loops"
>
> fluffy:positive list of what can be changed is needed, not the
> list of what can't be
>
> provide a list of changeable things that is *guaranteed* to
> work
>
> matthew: errors need to be specific on what is wrong
>
> [argument over what the current spec (JSEP?) says...]
>
> burn: many kinds of modifications will be needed before sending
> SDP out
>
> burn:what you send out does not match with what you set with
> setLocal?
>
> <ekr> who was speaking?
>
> matthew:jsep-02 has fixed the ice password issue
>
> fluffy:need to have the use cases that motivate changes
>
> justin:needs coming up in the future anyway...
>
> how much and what state does createOffer create? devices with
> HW codecs?
>
> <matthew> jsep-02 fixed the "createOffer is optional" language,
> though whether or not createOffer creates state is true or not
> is questionable
>
> <ekr> matthew: though, there's clearly an implication of that
> b/c of the validity window of the offer.
>
> <ekr> i.e. during the callback.
>
> <matthew> indeed
>
> martin:set down the list of things that MUST be possible to
> change
>
> burn:is everything else MUST reject?
>
> people have different interpretations...
>
> explicit list of things that MUST be changeable, explicit list
> of things that MUST not be changed and what falls in between
> the browser need to explicitly error report if it won't support
> it
>
> <martin> there is a list of things that we MUST be able to
> change; there is a list of things that MUST NOT be able to
> change, which trigger an error; everything else the browser
> MUST either accept or reject with an error
>
> <matthew> silent failure is incompatible with the assert on the
> 2nd slide. either the browser fires an error (invalidating the
> assert) or it is lying to you in order to make that assert.
>
> [the above three comments (markus, martin, matthew) try to
> record the consensus in the meeting]
>
> AndyHutton: Configuration/settings? how do they relate to the
> MUST, MUST NOT, ...
>
> fluffy:start with the list of use cases to get the
> MUST-be-changeable list
>
> matthew:listing MUST-items...
>
> [cullen taking notes...]
>
> adam: we will have APIs for some of these things anyway
>
> fluffy:if we have an API to control what createO/A gives, is
> that not enough, do we still need to change SDP for those
> "features"
>
> justin, matthew: there may be cases where SDP change is still
> needed
>
> fluffy: use cases are still not clear
>
> burn:do not worry about MUST-list anymore because anything on
> the MUST NOT list can still work...
>
> still sensing consensus on having three lists: 1. MUST be
> changeable, 2. MUST NOT be changeable, 3. (default) Browser
> MUST give an error if it does not support it
>
> fluffy:next issue
>
> when can two different video flows use the same m-line
>
> Proposal: all codec parameters are the same, "content-label" is
> the same, are in same MediaStream
>
> (hta, fluffy, martin debate the details)
>
> next: How does createOffer know to offer receiveOnly flow?
>
> want to receive video but don't have video camera
>
> justin has a proposal, mind to write it down here?
>
> matthew: can you put "send" in SDP before getting user consent?
>
> or do you first have to use receiveOnly and add sendReceive on
> a separate O/A
>
> ekr: how do we correlate multiple offered video m-lines to the
> multiple video streams the answerer has
>
> next topic: how does createOffer decide to offer a data
> channel?
>
> should OfferToExchangeData constraint be added?
>
> matthew: data is a great idea, but SCTP is horrible.
>
> fluffy:take this to the IETF
>
> tim:SCTP was decided in Feb
>
> <ekr> matthew: how do you really feel about SCTP?
>
> DanD:data is easy within a single app, but trapezoid between
> two apps is more difficult
>
> martin:issues could come also if the other device running the
> "same" app has constraints
>
> consensus: don't add this constraint
>
> next: DTMF
>
> will be discussed tomorrow with a proposal
>
> next: How long is SDP from createOffer/Answer valid?
>
> matthew:90 seconds would be an ultimate timeout
>
> use case: the SDP is sent to the server for modification
>
> should it be valid beyond the duration of the callback function
>
> ice candidates etc. time out in matter of 10s of seconds
>
> hta: time-to-live for the session description?
>
> matthew:can createOffer be called again after getting the
> modified SDP back from the server?
>
> matthew:proposes that createAnswer is valid only for the
> duration of callback and no longer
>
> consensus: it must be valid at least for the duration of the
> callback function
>
> ACTION: ekr to take follow-up to the list
>
> slides about rollback and error handling left for now
>
> (now lunch break until 1:30)
>
> Implementation status
>
> <fluffy> Cullen is scribing
>
> HTA is presenting what chrome is doing
>
> TURN and opus are scheduled to be in M24 all going well
>
> DTMF is waiting on this group
>
> Anant has a demo of firefox
>
> THe demo allows login, gives a list of users, then you can call
> one of the other users
>
> Have getusermedia, have peerConnection,
>
> expectation to not have behind a flag in firefox 19
>
> currently in firefox 18 in nightly builds behind a flag
>
> have a fairly complete version of the DataChannel
>
> showed cool file sharing with drag and drop using DataChannel
>
> THey have DTLS-SRTP
>
> ICE but no TURN
>
> prototype of Identity working
>
> Doing desktop first, then working on mobile
>
> VP8, opus, and G.711 as codecs
>
> HTA: There was a test web even 2 days ago
>
> dom: The idea of these is to get lots people to develop and
> contribute test cases
>
> … presentation of theory of testing
>
> … Event led to 404 test cases
>
> General error handling principles
>
> ScribeNick: JimBarnett
>
> [13]Anant's slides on error handling
>
> [13]
> http://www.w3.org/2011/04/webrtc/wiki/images/d/d1/TPAC_2012_WebRTC_error.pdf
>
> Anant: exceptions vs error callbacks. exceptions when you can
> detect error without blocking the main thread. Covers only very
> simple errors. In all other cases, error callback. So as
> policy, favor error callbacks because they don't block the main
> thread.
>
> Cullen: so lots of functions will do both?
>
> Anant: yes
>
> EKR: there's no way to get around exceptions occasionally.
>
> Tim: what do you do if you pass in something that is not a
> function as the error callback?
>
> Cullen: there are things that you should never see in
> production code, and those that have to be caught at runtime.
> The former are programming errors. We're interested in input
> errors.
>
> Anant: what goes in spec is a list of things that must be
> exceptions and what should be callbacks. This will be the
> result of consensus of browser implementations.
>
> Justin: so programming errors will be exceptions, bad SDP from
> the other side will be an error callback.
>
> Anant: exceptions will be only for development. They shouldn't
> occur in deployments at runtime. In exceptions should include
> name and message, which all platforms support We should
> standardize the names of exceptions, message can be
> platform-specific, thought should be human readable. stack and
> linenumber are useful where available. We will inherit from
> Error object to make SDPErrorObject.
>
> ekr: we should create new attribute for SDP error, rather than
> overloading linenumber.
>
> Anant: yes, having different names helps. We should standardize
> on the ones that need to
> be machine readable. Human-readable ones can be
> platform-specific.
>
> Dom: W3C policy is to re-use existing names as much as
> possible.
>
> Anant: we will use same object for error callback and
> exception. So name and message must be present in object passed
> to error callback. As with exceptions, human-readable
> properties can be platform-specific. For SDP errors may want to
> create a new property. Error callbacks should be mandatory. In
> current spec, they're optional. That makes it easy to make
> sloppy errors. If they're mandatory, at least you get an
> exception if you forget to define the error callback. It should
> never be the case that there's an error and nothing happens.
>
> ... In CreateOffer, exceptions: Invalid_callback,
> invalid_constraints, invalid_state.
>
> Dom: these can be webIDL type mismatch errors. We don't need to
> specify them separately.
>
> Anant: we can move invalid state to a callback (for cases where
> app violates the state machine that we define.) So we won't
> need to define any exceptions for (most?) functions. setLocal
> and setRemote can have invalid_sdp as error in callback. Assume
> that the success callback in set remote isn't called until the
> description has been fully applied.
>
> Cullen: if have setRemote with provisional SDP and then later
> will apply final SDP. Consider the case where are parsing SDP
> and acting on it as you go along. Do you have to roll back? It
> may be hard to do that.
>
> ekr: setRemote shouldn't generate callback until it's complete
> and in correct state.
>
> Justin: we need a separate error to signal case that media
> system is hopelessly busted.
>
> Cullen: we need two kinds of errors. Implementation will know
> whether the situation is
> hopeless or not.
>
> Harald: Invald SDP indicates that you have been able to
> rollback, so it's not fatal.
>
> Cullen: when SDP fails a syntax check and no state has been
> changed, as opposed to case where what you thought was a camera
> turns out to be a mouse.
>
> ekr: there are cases where the error is reversible, but you're
> still screwed.
>
> Harald: let's put Anant's proposal into the spec, and work out
> some of these details later.
>
> Anant: we can leave the decision on which errors are fatal to
> the UA. Further discussions of specific error cases on the
> list.
>
> Dan: are error callbacks mandatory?
>
> Dom: sloppy programmers can always put in a no-op error
> callback. On the other hand, if we make them mandatory, we will
> have to include them in our examples.
>
> RESOLUTION: ERROR CALLBACKS ARE MANDATORY. Anant to update
> spec.
>
> So decided that invalid_state will be a callback no exception
>
> <dom> [14]Error Types defined in DOM 4
>
> [14] http://www.w3.org/TR/dom/#error-types-0
>
> Decided that for SDP, will add a sdpLineNumber
>
> decided all errors callbacks will not be optional
>
> Call flows Walk-through
>
> Justin: when you call setLocal that's when UA starts gathering
> ICE candidates. Get callbacks as each candidate is gathered,
> another when all are gathered. If call createOffer before any
> candidates are gathered, it will have just your local address.
>
> Justin: if you want to do trickle, have to get offer first and
> then gather candidates. So must get getOffer callback before
> all candidates are gathered.
>
> Cullen: what things can cause gathering to begin? What
> indicates how many candidates to gather? How do you know when
> you're done.
>
> Matthew: it has to be a settable parameter whether createOffer
> callback can fire before you have enough candidates to produce
> valid SDP. I don't like to get a success callback when the
> offer is not valid yet.
>
> Cullen: one model is you can decide to wait until SDP is
> complete. Or we can say that you get callback immediately, then
> get another when you have all the candidates.
>
> Harald: must have ICE candidates to be valid.
>
> Cullen: but local address suffices for validity.
>
> Justin: in Jingle can send offer without any candidates and
> it's fine.
>
> ekr: it won't work with SIP.
>
> Adam: we should have matching examples and callflows, including
> trickle.
>
> Stefan: the Chrome implementation seems to work, with and
> without trickle. So it should be our reference.
>
> Justin: we discussed calling createOffer and setLocal early to
> start allocation, and then add streams later
>
> Cullen: we don't have mechanism for declaring dummy streams.
> What if had new method: gather n candidates.
>
> Justin: that's like setLocal with empty m=lines.
>
> Cullen: calling app has pretty good idea of how many candidates
> it will need. At least an upper bound.
>
> ekr: two proposals for pre-allocation: - a direct instruction
> to peerconnection, or some sort of dummy SDP via createOffer
> and setLocal.
>
> Cullen: we have 6 issues to discuss, all at least as big as
> this.
>
> Harald: I'd like to get an overview of all the questions. for
> this issue, we have two questions: how to know when SDP is
> sendable, and how to do pre-allocation.
>
> Matthew: what happens with resource reservation? What happens
> when you do all this gathering and find out that the camera
> isn't available any more?
>
> Cullen: next issue. How does receiving side find out what's in
> the offer, so that it can show the user: Alice is calling and
> she wants audio and video.
>
> Matthew: when you call setRemote, you have no idea how many
> times onaddstream will be called.
>
> Cullen: general model is a callback for each structure, and a
> final callback saying that you won't get any more. questions:
> when do you know that you've gotten all mediastreams? When do
> you know you have all the tracks? When do the callbacks fire
> with respect to the setRemote callback?
>
> ekr: at what point do side effects take effect w.r.t the
> callback? I think it must be before.
>
> Justin: when does peerconnection.remotestreams get populated?
> When you get a stream, the tracks should be filled in already.
>
> Matthew: parse them into the array, and then call
> processingcomplete.
>
> Justin: how about a callback saying the remotestreams value has
> changed? (It would be an array value, so this callback would be
> called only once.)
>
> Adam: it's more convenient to get the new stream in the
> callback, rather than having to parse the array to see what's
> different.
>
> Justin: but it's important to know when the changes are
> complete, rather than getting changes one at a time.
>
> State machines
>
> [15]Justin's slides
>
> [15]
> http://www.w3.org/2011/04/webrtc/wiki/images/f/fa/WebRTC_States_v2.pdf
>
> <inserted> ScribeNick: martin
>
> juberti: [presenting on states]
>
> martin: does back to gathering happen for moving from "relay"
> to "all"?
>
> juberti: you probably already have all the necessary candidates
>
> matthew: is there any way to remove candidates?
>
> juberti: ice restart
>
> ekr: can we have multiple components with one completed and the
> other one with no candidates
>
> juberti: we are in checking until all components have resolved
>
> fluffy: checking encompasses frozen
>
> matthew: restart doesn't affect active flows
>
> juberti: yes
>
> matthew: why not just describe each of the states as they
> relate to the underlying states
>
> fluffy: this is part of what I prepared for the last call on
> this topic and we rejected that
>
> hta: there is no harm in playing audio while video is failing
> and restarting
> ... is there any difference betwen connected and completed?
>
> juberti: middleboxes might require the updated offer that would
> be triggered from the completed transition
>
> martin: is this application driven or not, can the browser add
> new candidates and contine?
>
> juberti: restart is required by RFC 5245
>
> fluffy: the application is going to need to be involved
>
> juberti: if you get a new NIC (e.g. WiFi) you might just
> trickle that
>
> hta: transitions to starting are tied to user actions
>
> martin: how does the new WiFi candidate fit into this?
>
> juberti: that would trigger a transition to connected
>
> fluffy: there is an implication that disconnected might
> transition to failed, in the case where you were connected and
> you disconnect then something failed
>
> juberti: proposes changing name of "starting" to "noo"
>
> <hta> IceConnectState -> IceConnectionState
>
> juberti: remove onicegatheringchange, and provide just
> onicecandidate to fire when the gathering state is changed
>
> derf: propose to rename iceConnectState to iceConnectionState
> or something
>
> * missed the name conclusion
>
> ekr: is this same as the other state machine, just with the
> states merged?
>
> juberti: yes
>
> <hta> martin, name conclusion was to use IceConnectedState
> rather than IceConnectState (I think - my ears are going)
>
> matthew: set...(answer) can't be done twice?
> ... why isn't createOffer and createAnswer shown on the
> diagram?
>
> juberti: once you set...(answer), you may have removed some
> critical state for the offer, which invalidates some of the
> answers
>
> Summary of Action Items
>
> [End of minutes]
>
> ***************************************************************
>
> [1]W3C
>
> [1] http://www.w3.org/
>
> Web Real-Time Communications Working Group Teleconference
>
> 30 Oct 2012
>
> See also: [2]IRC log
>
> [2] http://www.w3.org/2012/10/30-webrtc-irc
>
> Attendees
>
> Present
> andy_hutton, adambe, hta, stefanh, derf, burn,
> dan_druta, richt, anant, dom, juberti, jim, matthew,
> ekr, fluffy, Magnus
>
> Regrets
> Chair
> Harald, Stefan
>
> Scribe
> DanD, Juberti
>
> Contents
>
> * [3]Topics
> 1. [4]Identity
> 2. [5]API for removing streams
> 3. [6]DTMF
> 4. [7]Other business
> 5. [8]What Triggers Candidate Fetching
> 6. [9]When is SDP sendable?
> 7. [10]Security for QoS labels
> * [11]Summary of Action Items
> __________________________________________________________
>
> <trackbot> Date: 30 October 2012
>
> Identity
>
> <dom> ScribeNick: DanD
>
> ekr presenting
> [12]http://www.w3.org/2011/04/webrtc/wiki/images/f/f2/Idp-issue
> s.pdf
>
> [12]
> http://www.w3.org/2011/04/webrtc/wiki/images/f/f2/Idp-issues.pdf
>
> ekr: Three proposals on the list
> ... Proposal 1 - Identity provided by gUM (Thomson)
> ... Proposal 2: Prompt user after call (Ohlsson)
> ... Proposal 3 - Site permissions with identity display (EKR)
> ... Proposals 1 and 2 require the user to explicitly assent to
> identity
>
> fluffy: How does the person answering the call who's calling
> before answering the call?
>
> ekr: Long term consent
>
> Mathew: I like to setup the video selection and enable the
> camera and the identity makes it complicated
> ... easy to be done with proposal 1
> ... We need to make sure the browser does not get into the
> address book without user's consent
>
> hta: I'm not happy with proposal 2
> ... when the green light go on
> ... I have an origin that is not bound to an identity but later
> get's bound
>
> ekr: Proposal 4 is a hybrid of 1 and 3
>
> dom: I don't think we should use a null value for a parameter
> with a different meaning as a parameter not being set
>
> ekr: I'm happy to use a different parameter name for option3
>
> justin: There's a difference between site having access to the
> camera and site recording
>
> dom: We're focusing too much on the green light where mobile
> devices don't have a green light; I want to make sure we don't
> build security on top of hardware indicators that are not
> always available
>
> martin: the idea is to have an indicator in the chrome
>
> justin: there's confusion about what the green light means
>
> fluffy: we are adding to the confusion
>
> ekr: the green light (the indicator) is supposed to be on once
> the camera is accessed
>
> hta: there is a issue when applications via USB can access the
> green light
>
> martin: if it's on can drain battery without sending any data
>
> fluffy: expectation is that when camera goes on light goes on
>
> ekr: going over the proposed rule
>
> dom: I'm a little bit confused. if the indicator is in the
> chrome I won't see it if I switch to a different app (on a
> mobile phone); how does that affect the reliability of granting
> access to a camera to a peer
>
> ekr: we should
>
> Mathew: will we be able to check for long term permissions?
> ... action on ekr to write something up on tainted streams
>
> hta: suggest camera access as a topic for the other issues
> later today
>
> API for removing streams
>
> hta: presenting what "remove" means
>
> fluffy: we should not look at index
>
> hta: either we don't remove the streams and you have a fixed
> index
>
> adam: if we reference it by object even if it's removed from
> the array I can still find it by object reference
>
> dom: this is what DOM is doing
>
> hta: in this proposal I have two methods to access: one by
> returning all and iterate another one by name
>
> fluffy: why are we handing developers indexes when they can get
> them by name
>
> dom: using index is not a bad thing as long as you don't assume
> that what it refers to is immutable
>
> adam: I don't think you want to have the sequence because
> developer can store the sequence and can change later
> ... the msid draft has already a proposal how to name things
>
> hta: we need to decide if we want to get rid of the indexes and
> go with the labels
>
> Mathew: legacy devices might not label
>
> Adam: we should go with ID's for both streams and tracks
>
> hta: we need somebody to write a proposal
>
> <martin> there seemed to be general agreement with Adam's
> suggestion, namely assigning ids to every stream and track and
> having label used only for human-readable text
>
> <dom> ACTION: Adam to update APIs to use mutable arrays of
> streams in peerconnection with ids [recorded in
> [13]http://www.w3.org/2012/10/30-webrtc-minutes.html#action01]
>
> <trackbot> Created ACTION-59 - Update APIs to use mutable
> arrays of streams in peerconnection with ids [on Adam Bergkvist
> - due 2012-11-06].
>
> DTMF
>
> hta: presenting API requirements
>
> hta: proposal to have two new functions on RTCPeerConnection
>
> justin: I like the idea of having the callback with tone and
> duration
>
> Martin: the proposal was to have a track (something that looks
> like a track)
>
> hta: what's the factory for that track?
>
> Martin: you construct it
>
> hta: now you need to the go from the track to the
> peerConnection
>
> Martin: you get the track from getUserMedia, you decorate it
> for DTMF and attach to the peerConnection
> ... on the receiving end it's simple
>
> hta: If I would be to implement where do I reliably place this?
>
> Fluffy: How do you know to negociate for DTMF?
> ... another one is long tones use cases
>
> justin: you can emulate this with the proposal
>
> fluffy: I"m fine with getting rid of the long tones
>
> Stefanh: I like the proposal. I'd like more control on the
> outgoing part
>
> burn: I think you can have the DTMF track be created by the
> peerConnection
>
> justin: I support hta's proposal
>
> hta: I'll modify the proposal to incorporate burn's suggestion
>
> Other business
>
> hta: reviewing the collected items for discussion
>
> What Triggers Candidate Fetching
>
> Mathew: Use case is when a web page displays "call agent" . I
> don't want all the visitors of the page to use my turn servers
>
> ekr: This should not be a problem
>
> hta: I don't understand why the use case is not satisfied
>
> Martin: you want to load the page and show as quickly as
> possible
>
> justin: we have two mechanisms to control the candidates
> gathering
> ... you can do that now
>
> Martin: this brings us to changing constraints on the fly
>
> fluffy: we want to preallocate now were' talking about how to
> do it in the API design
>
> justin: you would know what you need if you call setLocal
>
> fluffy: you can optimistically assume two
>
> Mathew: you got two use cases: the conference model where you
> call in to a conference or the public page. We need to support
> both efficiently
>
> fluffy: can we try to make a proposal
> ... it's less elegant but should work
>
> justin: we can do this using setLocal
>
> fluffy: I have a different proposal. Have a constraint that
> defines the preallocated streams
>
> ekr: it's fine for me
>
> hta: is this sort of creating a pool?
>
> fluffy: yes
>
> <martin> proposal is to add a new constraint
> preallocateCandidates, which takes an integer value that
> defaults to zero. setting this to any other value through the
> constructor or updateIce triggers the filling of a candidate
> set pool of that size. final actions are taken on
> setLocalDescription
>
> hta: decision to go with the proposal to create a pool
>
> <martin> cullen will take an action to follow up on the last
> issue (see my last item starting with "proposal")
>
> When is SDP sendable?
>
> <juberti> Next topic: when is SDP sendable
>
> <juberti> When do we know if we have all streams?
>
> <martin> my proposal for this was that the success callback
> would fire, at which time the array^Wcollection would include
> all the streams
>
> cullen: this doesn't let you know whether a stream was added or
> removed.
>
> matthew: that doesn't work in all cases anyway; imagine calling
> setRemoteDescription twice, the second time before all
> callbacks have been received.
>
> cullen: on all functions that have callbacks, you can't call
> the function again before the callbacks have dispatched.
>
> juberti: this would be setLocal, setRemote, createOffer,
> createAnswer.
>
> ekr: what about getUserMedia?
>
> … should be possible to ask for multiple cameras.
>
> juberti: I have a proposal to handle the getUserMedia case.
>
> adambe: what about onaddtrack events?
>
> adambe: do we need onaddtrack when a stream is added?
>
> cullen: I read the spec, I think it says any time the remote
> side adds a track, you need a callback.
>
> adambe: there's also an onunmute event on tracks too.
>
> adambe: how about only onaddtrack only when a stream is
> updated, as opposed to added/removed?
>
> juberti: I like that proposal.
>
> cullen: I don't.
>
> martin: Let's address the stream callbacks first.
>
> hta: Let's do that.
>
> martin: setLocal, setRemote, createOffer, createAnswer should
> all be non-reentrant.
>
> <fluffy> proposal is that that the createOffer / createAnser,
> setLocal, setRemote, you can not call the same function again
> if the callback from a previous invocation has not returned
>
> martin: during time between setRemote and callback, exceptions
> should occur on any of these 4 APIs.
>
> martin: when you call setRemote, stuff will happen, but the
> browser will return to stable state multiple times.
>
> <fluffy> On set remote, you install all the stuff, then does
> does callback for onaddstream for each stream, then does
> callback with null to on add stream, then it call the success
> callback for set remote
>
> dom: do any other APIs do this?
>
> anant: in XHR, some things are disallowed while it is running.
>
> martin: during state transitions, no other transitions are
> allowed.
>
> anant: you can't call open on the same XHR twice.
>
> stefan: might this introduce a timing problem? some browsers
> are slower than others?
>
> dom: I don't think so
>
> cullen: I would guess that this is an atomic change, and it
> takes some time - I would look at some API that has similar
> needs
>
> dom: maybe IndexedDB
>
> jimbarnett: call should block instead of throwing exception
>
> adambe: if you get onrenegotiationneeded, that could cause a
> problem
>
> cullen: onrenegotiationneeded fires after the success/error
> callback, if needed.
>
> martin: queuing of calls could work OK.
>
> cullen: one lock across all 4 APIs.
>
> <timpanton> The way most UI APIs deal with this is to say that
> those functions can only be called on the 'main' thread
>
> hta: proposal is: when calling one of these APIs - check args,
> throw exception if needed, check lock, do work or queue task if
> needed.
>
> <timpanton> if the callback is also invoked on the same thread,
> this makes the thing queue naturally.
>
> hta: nobody argued for throwing an exception instead of
> queueing.
>
> hta: anant to write up the details here.
>
> <dom> ACTION: Anant to write up queuing mechanism for
> set{Remote,Local}Description, create{Offer,Answer} [recorded in
> [14]http://www.w3.org/2012/10/30-webrtc-minutes.html#action02]
>
> <trackbot> Created ACTION-60 - Write up queuing mechanism for
> set{Remote,Local}Description, create{Offer,Answer} [on Anant
> Narayanan - due 2012-11-06].
>
> <martin> callbacks have to be called on the same thread, there
> is only one thread
>
> juberti: when does onaddstream fire?
>
> <timpanton> So how could you get re-entrancy ?
>
> hta: onaddstream fires after installation is complete, but
> before the success callback has been dispatched.
>
> <martin> re-entrancy applies only in the sense that the actions
> associated with the methods take time and so could
> (conceivably, without these measures) operate in parallel
>
> burn: no add stream for failures, naturally
>
> <martin> the actions occur on browser-internal threads or "in
> the network"
>
> <timpanton> ok. got it.
>
> juberti: when is pc.remoteStreams updated?
>
> adambe: before the first onaddstream callback is fired,
> remoteStreams will be fully up to date with the changes.
>
> derf:<same thing>
>
> cullen: stream names are confusing.
>
> derf: event callbacks need to change to be less confusing.
>
> <dom> ACTION: Timothy to write up proposal for new stream event
> names. [recorded in
> [15]http://www.w3.org/2012/10/30-webrtc-minutes.html#action05]
>
> matthew: if SessionDescription is 3264 SDP, that SDP must
> always be 3264-compliant.
>
> cullen: if you can't do local candidates, we should return an
> error when trying to write SDP.
>
> matthew: will trickle update 3264?
>
> cullen: I think we'll need to.
>
> matthew: if we're doing trickle, we're going to need to update
> something.
>
> matthew: how do we generate workable SDP when trickling?
>
> matthew: Chrome currently generates broken SDP?
>
> matthew: How do I get valid SDP in the non-trickle case?
>
> cullen: wait for ICE complete callback, then SDP will be fully
> filled-in.
>
> matthew: but what about the initial setLocal? That SDP isn't
> fully valid.
>
> juberti: That is just a subset of the trickle case.
>
> matthew: But the SDP still isn't valid.
>
> juberti: We are going to solve this with the trickle ICE I-D.
>
> … and then the rest of the stuff should fall into place. You
> can call setLocal with an initial "no candidates" SDP, and then
> gathering commences.
>
> hta: we'll refer this to the IETF rtcweb WG to figure this out,
> and then we can resume this discussion.
>
> Security for QoS labels
>
> matthew: Want to see API where packet priorities can be set.
>
> hta: culler's proposal does this - gives 3 levels of
> priorities.
>
> cullen: API that provides 3, 4, etc levels
>
> matthew: I don't know which one is more important.
>
> matthew: data could be above or below media (gaming, higher,
> file transfer, lower)
>
> dand: where would this priority be set from an API perspective
>
> stefan: on a track or datachannel.
>
> <dom> there was a (currently abandonned) proposal for
> setPriority on XHR that also had 4 levels FWIW
> [16]http://ajaxian.com/archives/xmlhttprequest-priority-proposa
> l
>
> [16] http://ajaxian.com/archives/xmlhttprequest-priority-proposal
>
> matthew: someone needs to write up a proposal for where these
> things go.
>
> hta: set at initialization, or during the call?
>
> matthew: I think it could be initialization.
>
> dand: once we get this request, we need a confirmation from the
> browser that it tried to accomplish this.
>
> martin: why?
>
> dand: this request can be fulfilled by going to a policy
> server.
>
> matthew: I don't care about marking, I just care about the
> congestion control prioritization
>
> matthew: not all packets will be labeled the same way
>
> matthew: either for different streams, or different packets in
> the same stram
>
> hta: need to set priority levels, and have it per track, and
> have 3 or 4 levels.
>
> hta: this will set congestion control/queuing in the browser,
> and setting of QoS is something for further study
>
> matthew: cullen has already written a draft
>
> goran: cullen's draft refers to other drafts
>
> cullen: we will remove that reference.
>
> cullen: we don't want JS to set the diffserv code points, but
> we do want it to be able to discover them.
>
> hta: want mapping from track to 6-tuple?
>
> <juberti> ACTION: Cullen to update draft to remove reference.
> [recorded in
> [17]http://www.w3.org/2012/10/30-webrtc-minutes.html#action06]
>
> <trackbot> Created ACTION-61 - Update draft to remove
> reference. [on Cullen Jennings - due 2012-11-06].
>
> Action. stefanh to propose priority API.
>
> <juberti> ACTION: stefanh to propose priority API. [recorded in
> [18]http://www.w3.org/2012/10/30-webrtc-minutes.html#action07]
>
> <trackbot> Created ACTION-62
>
> Adjourning for lunch.
>
> Summary of Action Items
>
> [NEW] ACTION: Adam to update APIs to use mutable arrays of
> streams in peerconnection with ids [recorded in
> [19]http://www.w3.org/2012/10/30-webrtc-minutes.html#action01]
> [NEW] ACTION: Anant to write up queuing mechanism for
> set{Remote,Local}Description, create{Offer,Answer} [recorded in
> [20]http://www.w3.org/2012/10/30-webrtc-minutes.html#action02]
> [NEW] ACTION: Cullen to update draft to remove reference.
> [recorded in
> [21]http://www.w3.org/2012/10/30-webrtc-minutes.html#action06]
> [NEW] ACTION: stefanh to propose priority API. [recorded in
> [22]http://www.w3.org/2012/10/30-webrtc-minutes.html#action07]
> [NEW] ACTION: Timothy to write up proposal for new stream event
> names. [recorded in
> [23]http://www.w3.org/2012/10/30-webrtc-minutes.html#action05]
>
> [End of minutes]
>
>
>
>
>
Received on Thursday, 15 November 2012 14:18:44 UTC