Web Real-Time Communications Working Group F2F Meeting

<trackbot> Date: 07 February 2013

<dom> stefanh: propose to adapt agenda in light of lack of progress on SDP interface yesterday

<dom> ... instead of SDP agenda item, we'll look at video scaling and cropping in Media Capture Task Force

<scribe> scribe: timeless

Conclusions Media Cap

[ Conclusions Media Cap discussions Boston ]

[ hta reads slide ]

hta: any comments/complaints?

[ None ]

cullen: thanks guys for doing this

MediaStream Cropping

[ MediaStream Cropping ]

justin: this is a quick set of slides i put together
... for how a camera could stream could be provided in an aspect ratio not equal to native

[ Problem statement ]

<dom> Justin's slides on aspect ratio

justin: most sensors and cameras are moving to 16:9 capture
... not all cameras are new
... quite a few are 4:3
... it's hard for a full screen gui, especially full screen gui @ 16:9
... <video/> will always pad to fit
... as opposed to crop to fit
... even if <video/> would crop to fit
... you wouldn't want to encode those bits that'd be cropped at the other end
... if you're doing local preview, you'd probably want to see the other side
... so people can see if they're out of from

[ Solution: crop 4:3 to 16:9 ]

justin: discard top/bottom (or left/right if needed)
... Flash's camera api would do this if you asked for an unsupported res
... instangram likes squares, so you could get this
... i'm only concerned w/ crop - letterboxing/pillarboxing are available from <video/>

[ General Approach ]

justin: opt in, new mandatory constraint
... you want this specific height, width
... width.max,min=640
... height.max,min=360
... allowCrop=true
... if camera supports resolution that falls within mandatory constraints, use that res
... if camera supports res that exceeds mandatory constraints, allowCrop=true to crop satisfy mandatory constraints

dom: if this is mandatory
... how does it fit?
... it doesn't seem like a `selection` critera

justin: it's more for having chosen a camera
... selecting settings later

dom: so, it's a constraint that may be more for control than selection

justin: i'm intending it for that, yes

burn: this is control, not selection
... why mandatory { allowCrop }
... instead of mandatory/optional { cropIfNeeded }

justin: for optional things

<dom> [this seems to point to a growing divergence between constraints-for-selection and constraints-for-setting]

justin: if these bounds are not hard bounds
... then it'd be a closest match

burn: if it didn't get it, it didn't get it
... the point of optional constraints is notifying them if they can't be satisfied
... i'm proposing `cropIfNeeded`
... if you put it in mandatory, then it would crop if it needs to and can

justin: do you have an example?
... i have a camera that can or can't crop in hardware

burn: you might have other optional constraints relating to aspectRatio
... if you get those, you might want cropping,
... if not, you might not care

JonLennox: comment, that might make this harder
... i'm often in a situation of VGA camera
... cropping `middle 2/3` makes it easier
... but on a mobile phone in landscape, cropping like that doesn't do what i want

justin: i have another proposal
... PeterThatcher mentioned that problem on the ride over

fluffy: trying to simplify things
... i assumed crop was always true
... if they asked for 16:9 and the input is 9:16, they get a postage stamp
... person rotates phone, problem goes away
... caution that adding complexity won't help people
... what happens if it's mandatory and it comes back `can't`

hta: i kind of like the general approach
... but i dislike the examples
... i live in the world of resizable windows and interchangable cameras
... i want to write a UI once
... that will work when the user resizes the window
... and works w/ Barbie Doll camera
... and an HD camera
... i want 1 of 2 things
... either everything in picture field
... or i want picture to fill my field
... in neither situation should we stretch the picture
... remove the talk about min/max, and use the res.
... if you need to crop, crop

justin: are you +1'ing fluffy ?
... if you ask for a res, and

hta: when media is flowing
... browser knows where it's going
... and it knows where it's coming from
... it will have to do a fit operation
... this is an instruction to browser to crop instead of letterbox
... it doesn't instruct browser to aim for res

timeless: he's complaining about the example

justin: he's saying constraint pillarbox/crop

hta: my real window size is 250:172
... it should fit to that
... the best way to achieve that
... i think the browser should do what it needs to do

stefanh: with <video/> it can adjust to video aspectRatio
... and there are examples of using <canvas/> to crop
... canvas uses lots of power
... and you lose advantage of sending non-displayed bits

justin: i was going to add <canvas/> in an earlier slide
... if we had a Media Stream Processing API

[ laughter ]

justin: doing things in the GPU
... could be very efficient

derf: i want to apply this to any video stream
... not just cameras
... according to settings, a <video/> can't have this applied

[ Examples ]

justin: this isn't the alternate proposal
... just how to use this constraint
... 720p sensor, want 4:3

width.max,min=960

height.max,min=720

allowCrop=true

burn: on this topic of do we let browser do its wisest thing
... or let JS dev give a preference
... i don't think it makes it more complex to allow both cases
... it's quite productive to let the user choice
... crop, pad, best-guess

justin: that's what i heard fluffy say

burn: if there's a mismatch between <track> request and destination
... then you can express a preference as an app writer
... for cropping, or padding, or not caring

justin: how do you recognize this mismatch when the destination is PeerConnection

timeless: shouldn't PeerConnection be able to express a resolution?

burn: there's no way to express PeerConnection res as you do for <video/>
... unless we want to add that

[ Yes ]

burn: that would address a lot of this
... because then it becomes parallel to <video/>

fluffy: what i heard by hta was what i said
... a resize by Pretty, Ugly, ...
... for something that crosses the PeerConnection
... if we want that to propagate back to the origin
... the resize will have to go back across the PeerConnection
... involving a renegotiation
... complicated, but very nice
... if no one sets constraints
... but we need a way to say "i'm going to send 640x480"
... by creating a track with that
... and PeerConnection says "oh, that's what i'm sending"
... and then it doesn't negotiate

justin: someone changes window size
... and that pushes it all the way back
... if someone changes window size and camera has to stop and restart,
... that yields wacky behavior

fluffy: if camera is SIF camera
... and someone selects QCIF
... allow crop also
... have constraints?

justin: proposing

<dom> s/QSIF/QCIF

justin: PeerConnection encoder can do cropping internally

<dom> CIF on wikipedia

justin: the point is to give something to the encoder to do efficient scales
... for power of two scales
... for efficient matches of what output can display

martin__: the more i think of this
... the more i think i have too many ways to do the same thing
... this seems to belong in the renderer
... we have <video/> it may pillarbox/letterbox/crop
... if we treat PeerConnection as renderer
... then i don't think we need constraints

justin: assume a single camera
... no choosing
... for WYSIWYG
... do we have 2 mechanisms or 1 ?
... same way to ask camera for get X res, same way for X' res

martin__: two ways
... @ <video/>
... and a way @ Track()
... which may or may not
... if each sink is the only place to set these properties
... the only constraint would be for selection
... i'm coming to the conclusion that that's pointless
... since in most cases it's orientation of camera that matters most

hta: device selection is different topic

ekr: whatever camera aspects
... if i plug in arbitrary res
... to <video>
... we expect something as reasonable as possible

[ it will rescale in letterbox or pillarbox ]

martin__: there's no way to control it

justin: is it specified?

stefanh: i think it's specified
... you can ask it to auto grow

justin: you can never crop

ekr: anything really sophisticated
... like Media Stream Processing API
... want something analog to <video/>
... i want to force fit this camera to this stream
... this is a stop-gap?

justin: fixing a common problem w/ an efficient+cheap solution

ekr: i could live w/ either @source or @sink
... i think i exploded at pushing across Connection

stefanh: to martin__

<dom> HTML5 spec says: "In the absence of style rules to the contrary, video content should be rendered inside the element's playback area such that the video content is shown centered in the playback area at the largest possible size that fits completely within it, with the video content's aspect ratio being preserved. Thus, if the aspect ratio of the playback area does not match

<dom> the aspect ratio of the video, the video will be shown letterboxed or pillarboxed. Areas of the element's playback area that do not contain the video represent nothing."

stefanh: to add Crop <media> element?

martin__: yeah

burn: i think simplest logical model for all of this
... is your Track is used to identify what's natively produced
... and Sink is what you expect to get out
... and influences where you munge
... so PeerConnection should have things like <video>
... i think constraints on Track still make sense
... so you can ask for High Res from Camera (which offers High Res and Low Res)
... so an app writer can say "these matter to me"
... so sink does processing
... and make it clear that's where it is

justin: so that's a proposal to flag it
... at ...

burn: allow back propagation or not is up to us
... we could allow settings on PeerConnection to allow settings to go across
... PeerConnection gets to decide how its properties as a sink are set

justin: choosing how a device is opened up
... i argue this is analogous
... saying "give me X", or saying "give me X'"
... we could talk about feedback mechanism
... saying `camera can produce X res's`
... the reason allowCrop=true is there
... is for if you're in Portrait mode
... and you don't want to do cropping
... maybe apps don't ever do that
... give me these pixels no matter what
... give apps some flexibility

burn: i'm not recommending taking away control from app writer
... question is where to do it
... on Sink or on Track

PeterThatcher: is it safe to say we want to avoid reopening camera?
... does app tell browser this?

justin: most flexible is to open @high res
... and crop/scale
... cameras have non-0 reset time
... and there's noticable artifact as they stop/restart camera

PeterThatcher: would support controlling the crop at camera open time to avoid other constraints changing and causing re-open

<dom> ScribeNick: dom

justin: changing camera after feedback from sink

martin__: the simple response to that is: don't do that
... if you're operating in the wrong mode for the display you're opearting in, you'll have to deal with it at some point
... This proposal is adding processing in the pipeline, which is what I wanted to avoid
... leaving constraints for elements that matter for the hardware
... anything related to processing should be left up to a processing api
... that api would provide a way to crop the video among other things

justin: that would be great, but that expands the scope of needed work

martin__: we would need to gather use cases, but that would be much cleaner than trying to push processing through constraints

adambe: everytime we talk about constraints, it sounds like this could be for sources, tracks, or sinks
... that is very confusing
... it would be easier if only of these should be the focus of constraints

burn: I have a similar comment; I think it should apply to two, not three

adambe: we have the recorder as a sink, for peerconnection, stefanh proposed a transport handler to support setting priority etc
... for the MediaElement, you can control the width and height, but I don't know how you would control framerate, etc
... It seems more natural to let the sink in control

Dan_Druta: I heard two aspects of applying constraints for: efficiency, and user experience
... we should be very clear about it when we talk about pros and cons

juberti: re the sink driving, that's an important model, but I don't think we should force everyone under that model

gmandyam: in the media capture call in December, we discussed @@@
... you can set options for recording independently of the source
... it's not a real-time operation either, so with different considerations than peerconnection

juberti: setting stuff at the source level doesn't invalidate options at the recording level

burn: the current state of the settings proposal that we talked about, and the world we live in the current spec
... is that of source, tracks, and sinks, you can only set constraints on tracks and sinks
... peerconnection doesn't have the same kinf of settings capabilities
... there is a need to control what the source provides natively
... (if the device cheats e.g. with Apple, there is nothing we can do about it)
... we only two, but we have three places
... we could set constraints on sources, tracks, sinks
... the problem with constraints on peerconnection, you can have multiple tracks going in a peerconnection
... each of these tracks could have their own configuration
... this is why we've been talking about control on tracks
... the problem is to determine whether munging is done at the track abstraction level (and the source is where you set what you want), or at the sink abstraction level (and the track is where you set what you want)

fluffy: +1 to burn
... we're lacking a mental model of all this
... there are sources, sinks, and tracks connect them together
... the problem with sinks is that we don't have much control about it

<gmandyam> @@@ = At December MediaCap TF call, the participants agreed that width-height combo settings are required for the recording API. I proposed a possible setting to the mailing list.

fluffy: as a result, tracks are preferred location for exercising control
... if you put constraint on tracks, that affect what's produced by that track
... for the sources, the general model is that they look at all the tracks they deliver, and they select the best setting that allow them to satisfy the tracks they're producing
... As a result, the mental model should be that Tracks are where we want to set constraints preferably

juberti: that makes sense to me

Josh_Soref: regarding the risk of latency attached to reopening a camera
... could this be something we can prevent using a constraint?
... (preventing from reopening a camera to adjust to a new setting)
... On the PeerConnection bit, I've always assumed @@@

stefanh: I think we need more control over transmission media
... we could apply that to tracks, or provide an api in peerconnection per-track

juberti: I would prefer to apply it via peerconnection

stefanh: I agree

hta: the video element has the CSS3 property to define the cropping/fitting model

martin__: I don't mind fluffy's suggestions, except that it means you now have two ways to control this for the video element
... I'm kind of tempted that we don't have this setting API for that
... I'll have to talk with Travis about that

juberti: I have an alternate proposal

<hta> (the css3 property is called object-fit - it took me a while to find it)

juberti: where you define the maximal crop aspect ration

-> http://dev.w3.org/csswg/css3-images/#object-fit object-fit in CSS Image Values and Replaced Content Module Level 3

Josh_Soref: in the recorder case, there isn't necessary a post-processing server somewhere else
... I can be doing local recording

juberti: the cropAspectRatio is the process aspect ratio

@@@: we would make it clear that process stuff would only be used for peerconnection

cullen: do you have a slide on scaling?

juberti: no

cullen: I would love that we say we never scale up

martin__: disagree, it has to happen in some cases
... I think we're talking about processing, and where that processing is applied

stefanh: to avoid upscaling, we would have to change the mediaelement, since it scales up

cullen: I was referring to not allowing to scale up at the track level, but I think the length of the line shows I'm losing on that one

ekr: what does it mean to scale up a track?

cullen: say you've acquired a sd camera
... and then you apply a constraint to the track to get a bigger resolution

ekr: so I have a SD source that gets a SD track, plugging it into an HD video tag — that will be ugly
... if you were to apply to same setting to a peerconnection, the same thing would happen
... what you're saying that we should not allow to augment the resolution of a native track

burn: I think you should have control
... but fluffy, you're badly schizophrenic
... if the constraint you set on the track is the output of the track
... and it's ok for munging to happen
... then scaling up has to be allowed to happen
... what we would want is to have constraints to limit the way the processing can be done
... and I think that's what justin is describing

juberti: the goal is to provide a simpler way to do processing without providing a full-blown graph processing api
... I worry about an authoritarian model where @@@
... I don't think it makes sense to send the quadruple amount of bits with no increased quality

tim: afaict, no browser has implemented object-fit property except opera in the past two years

-> http://caniuse.com/object-fit object-fit in caniuse

jan: we're a bit off-track with upscaling
... we've talked about source constraints and track constraints
... if we had peerconnection.maxWidth, peerConnection.maxHeight, this would cutshort this whole debate

juberti: I think the peerconnection should have specific control on what each track aspect is
... I think we're pretty far afield to specific problem I'm trying to solve here

martin__: if you're concerned about sending more bits over a PeerConnection than is necessary,
... having these settings on PeerConnection allows the PC to say "you've set this higher than what the source provides"

Jan: I would modify my earlier proposal to have PeerConnection say the aspect ration of what the client will care about (e.g. 4x3)
... this is only for peerconnection

juberti: not really, since object-fit is not implemented
... some cameras can do cropping at the hardware level, but not all of them
... this would let us solve the problem to these various cases
... it's not the general mediaprocessing api, but it has a better RoI

hta: I think there is some level agreement that cropping needs to be able happen
... and the application should be able to have some control over it
... there is a fair bit of debate on what the model is on what constraints apply to, and how they show up at the destination of the track
... vs modification at the source of the track
... We kind of passed that boundary when we agreed to be able to produce multiple resolutions from a single source
... I would like to ask martin to write up his concerns on setting constraints on track
... and propose an alternate approach

<hta> ACTION: Martin to write up his concerns with using constraints on track to manipulate stuff that might apply to sources and might apply to sinks [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action01]

<trackbot> Created ACTION-83 - Write up his concerns with using constraints on track to manipulate stuff that might apply to sources and might apply to sinks [on Martin Thomson - due 2013-02-14].

hta: I would like justin to take an action item to describe this constraint in a way that can be included in the document

<scribe> ACTION: Justin to describe the constraint for aspect ratio for potential inclusion in the document [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action02]

<trackbot> Created ACTION-84 - Describe the constraint for aspect ratio for potential inclusion in the document [on Justin Uberti - due 2013-02-14].

Error handling

<timeless> scribe: timeless

hta: there are cases where we need to provide more information than just the error name
... e.g. when there is an sdp error
... we want the line of where the SDP arose

<inserted> ScribeNick: timeless

hta: could people help me refresh my memory?

burn: we were trying to finalize the principles
... but we decided we needed to look at specific
... and Adam Roach said we needed to hear from implementers
... as they implement
... instead of "ooh, maybe it will do this, maybe it will do that"

ekr: what errors we're seeing
... what you need to return
... how often, and when
... justin and i are debating Error1, or Error2 and Error3
... i mostly see malformed SDP
... a browser screwed it up, or someone mangled it
... a parser of why i didn't like it and what it was
... Adam Roach pointed out the same need for ICE
... when i look at existing ICE information
... that would be a long way toward knowing what's going on
... a way to detail every check
... would be useful
... very confusing errors
... the only thing in the docs at all is detailed ICE info

justin: for Chrome 25, we'll have which line of SDP it choked on
... SDP people complain about Dom11 for not having crypto lines
... we should see if that's useful
... we also created chrome:ice-debugging (not for JS)
... to allow people to debug apps w/o exposing js apis to apps
... we'll have more debugging tools
... before we invent too many more things

ekr: most of these errors are programming errors
... i think it's a great idea to have inspectors that are only accessible to programmer

<dom> [The WebSockets API is very careful in not exposing too many details about errors: http://www.w3.org/TR/websockets/#feedback-from-the-protocol ; not sure if there are similar risks in our case]

ekr: you test when you write code
... you don't know why
... once you get it working, it works
... ICE check failed on component 1 to the user
... is unlikely to help the user

fluffy: we're putting your email address in our errors
... i don't think we've reviewed the doc for errors that can come up
... we're seeing implementations
... a port of Firefox that also does 264
... not being able to report up mismatches
... problems that you don't see since you only do one codec

derf: how we report errors is good
... we need to define what happens when there's an error
... you might parse SDP, it might be valid, but not apply to session state
... mismatched m=lines between offer and answer
... i don't care about behavior, but it needs to be consistent
... we could Tear down the entire call
... we could not access the entire SDP
... or browser crashes
... but it shouldn't be inconsistent between browsers

dom: WebSockets API is careful
... about avoiding exposing details

ekr: SDP exposes the entire Network Topology

<dom> [so should we drop SDP? >:]

ekr: hta's proposal of statistics
... could be useful
... or if i'm showing my video instead of remote
... if there's no RTP coming in
... re derf's proposal
... there's going to be a whole class of things
... passing bogus SDP
... sometimes that's a parse error
... sometimes that's an application error
... standards should encourage right behavior
... but if you get halfway through applying an SDP
... i can't roll-back
... it might be better for me to close

juberti: someone had a action to describe how rollback should happen
... two sets of errors
... @SDP layer
... either not eat
... or tear down
... or @SRTP
... and we could blow up in Security layer
... we can make best practices

hta: implementer feedback
... some parse errors
... are easy, some are harder to pin down
... the missing crypto line is hard
... which line is the one to report it for?
... it would be nice to have some descriptive text
... not for applications
... but useful for debuggers

<dom> [oh boy, are we now describing an SDP parser API?]

hta: or maybe useful advice

ekr: missing crypto line v. missing fingerprint
... having looked at jingle
... it wouldn't be at parse time
... it'd be later
... you need a textual report
... "i'm sad for reason: ..."
... parsers print to logging but don't propagate

juberti: that landed last night
... in Canary

stefanh: no big need for adding errors
... but more on what happens when you encounter an error

hta: this is part of procedure description for setLocalDescription/...
... we left off precise language
... on the idea of getting the general principles before we write precise language
... i think it's the time to write precise language

[ Coffee Break for 15 minutes ]

Data Channel API

jesup: W3 flavor of data channel discussion
... i hope less controversial than the IETF flavor
... maybe we'll make a decision

[ DataChannel JS API ]

jesup: feedback from Mozilla Devs working on those apis
... has been generally positive
... it's worked the way they've expected
... primary unresolved area
... is how to create data channel and how it interacts w/ createOffer/createAnswer

[ Changes to API ]

jesup: IETF 85 ... protocol to add data channels to agree on interop

martin__: the web socket idea of a sub protocol?

jesup: i guess so
... an IANA registered string
... if you do this, you agree you're exchanging in some form
... since we're mirroring WebSocket
... this is that thing from WebSocket

adambe: latest version of WebSocket lets you say you accept multiple protocols?

jesup: but you can only use one once it's in flight

<dom> WebSocket Subprotocols registry

jesup: our way to create data channel is different from WebSocket

martin__: no need to negotiate
... it's with itself

jesup: you would need a way to signal back agreement, complexity for no reason

[ Proposal ]

jesup: like yesterday, but w/o SDP
... all the JS layer
... no DataChannel unless you call pc.createDataChannel()
... you can call it any number of times before createOffer/createAnswer
... if called after initial createOffer, it triggers a negotiationneeded

hta: createOffer or setLocalDescription?

jesup: before SDP or after SDP is the distinction
... relative ICE things

hta: what if they createOffer throw it away and call createOffer again?

jesup: maybe `initial` is wrong

hta: one solution is first call triggers negotiationneeded

jesup: createStream throw away stream, stream is still there
... same for data channels

martin__: seems like there's some need for ...

derf: we had an AI for when negotiationneeded fires

martin__: and when other things fire after

[ Problems ]

jesup: unless you sniff SDP, you won't know if there was a data channel

<martin__> the question is a) when onnegotiationneeded fires and b) when it clears such that it could fire again

jesup: we can't add an m=line in an answer
... if you want it, you'd need a second round of offer-answer

[ Example ]

<derf> martin__, I suspect the answer to b) is that it can always fire again.

[ Decisions! (Please!) ]

jesup: use createDataChannel to trigger m=application line
... if needed, createDataChannel will call negotiationneeded

martin__: need to be >0 open channels to include m=application
... because you could create a data channel, close it, and then createOffer

hta: i'm fine w/ this
... is that what you want?

jesup: basically

XXX

paul: namespace for sub protocols?
... subordinate to webrtc channel namespace?
... should be same namespace as websocket subprotocols?
... i don't know that

<martin__> ACTION jesup to if we replace onnegotiationneeded conditions to the same as those we have for addStream()

<trackbot> Error finding 'jesup'. You can review and register nicknames at <http://www.w3.org/2011/04/webrtc/track/users>.

jesup: i think it's an IANA register

paul: just want a procedure

<martin__> ACTION Randell to make onnegotiationneeded conditions the same as those we have for addStream()

<trackbot> Error finding 'Randell'. You can review and register nicknames at <http://www.w3.org/2011/04/webrtc/track/users>.

paul: to define these

TedHardie: it's a first come first server registry
... set up by WebSocket folks

paul: single level name

hta: you can define protocols for data channel
... that don't work well w/ web sockets
... that might be an argument for different registry

jesup: not an argument for different registry, just don't use w/ websocket

TedHardie: for this room, push to IETF side

jesup: i agree
... what `protocol` means shall be defined by IETF

<martin__> OK, randell isn't a registered user

Dan_Druta: protocol isn't limited to registry, right?

jesup: if it wishes to ignore the registry, nothing can force it
... I yield the remainder of my time

stefanh: we made a decision to go w/ this w/ hta's amendment

<dom> ACTION: stefanh to get Randell to make onnegotiationneeded conditions the same as those we have for addStream( [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action03]

<trackbot> Created ACTION-85 - Get Randell to make onnegotiationneeded conditions the same as those we have for addStream( [on Stefan Håkansson - due 2013-02-14].

Stats

[ Stats ]

[ hta defines Microphones `chair at desk`, `speaker at wireless`, `individual at center mic` ]

<scribe> [ New API ]

hta: changing datatype of what comes back on stats report
... to be an object that's able to tell you the identifiers
... and able to get those objects
... each object has type and id
... and set of names, and a getter to get value for name

juberti: like a Dictionary?

<scribe> [ New Objects ]

hta: exactly
... Dictionaries are a name value mapping
... I decided this is a fine way of putting stuff into documentation
... that can be checked with a WebIDL parser
... sequence returned order is somewhat curious
... depends on alphabetical names
... and based on hierarchy
... things defined in derived dictionary are added to the sequence (sorted) after the original dictionary's list
... this is a pain in the posterior
... i looked into the state of a Dictionary
... and how to return one from a function
... the answer i got is that `it's doable, w/ 3 experts in DOM state manipulation`
... so i propose no dictionaries
... [as from the previous slide ]

[ Principles of object definition ]

hta: use Dictionary as documentation method
... pointers are done via ID
... Keep it simple (and well defined)

martin__: i'm unsympathetic to the hard to implement

hta: did you implement something that fully implements dictionary

martin__: one thing i didn't deal w/ was extra params
... the idea that you'd have trouble passing objects to js as hard to do...
... we've discussed this at length
... the things we're getting are key-value-pairs w/ primitives and maybe arrays
... but you're forcing them to use unfamiliar constructs

adambe: JS has a way to deal w/ KeyValue pairs
... couldn't we say we're returning a JS object w/ these values
... and not promise order
... that gives us enumeration and lookup from JS

hta: one vote for Dictionary
... and one comment for Object

fluffy: in favor of making this work

<dom> [I think you can use [NoInterfaceObject] interface to describe regular JavaScript objects]

fluffy: keen to restrict semantics to that of Dictionary
... if function calls get mapped to thing, this would be a disaster
... nearly every language has something like Dictionary
... I want Dictionary semantics, ... ASN.1 - i don't care

ekr: is there an argument against these being properties
... that isn't rooted in implementation details
... why isn't names() and getValue() just properties of the object
... one is that it's Lousy for JS user
... and one that it's inconvenient for the C++ WebKit dev
... is it just PITA for WebKit?
... i'd have expected that a param was property index (.keys)
... and reference by .<> or [<>]

fluffy: you expect Dictionary

ekr: what's the merits of this other than implementation?

juberti: if it was []/. instead of getValue()

dom: i understand Dictionary is a pain
... but why not Interface
... this is similar to GeoLoc in a way
... they use [NoInterfaceObject]
... we shouldn't define our own way of declaring our own local data
... WebIDL w/ interfaces provides that

martin__: use partial interfaces

hta: partial interfaces w/o source

juberti: why objects
... couldn't we make them more like regular dictionaries?

hta: if `void (RTCStatsReport statsReport)` with a map
... Dictionary provides some guarantees and constraints i don't want

martin__: it doesn't need to be a Dictionary
... it could be [] or .
... you can do this in JS in browsers w/ handlers
... i don't see a problem w/ implementation
... if you aren't backing directly w/ C++
... we need a map

adambe: `statsReport`
... report is a complete thing
... and you have subreports
... you want two levels
... you could guarantee a timestamp, type, id
... why are those on a different level?

martin__: individual stats object could be created at different times

adambe: move timetamp/id into getValue?
... i'm disregarding the top object

[ RTCStatsReport ]

adambe: i don't see why we need a stats.type v. stats.getValue(x).

hta: how do you specify that in WebIDL

martin__: i'd rather show example code for exact number of packets

dom: I think in WebIDL, your additional data structure inherits from a core interface
... Object is a Map

fluffy: don't drive from WebIDL
... figure out from how you want to use it
... i think it generally does

hta: i don't have slides w/ code
... if i have dictionary foo { int blah; }
... i can't have additional attributes

martin__: no one sees the dictionary type
... so you can have dictionary privatefoo : foo {...}

hta: they are unobservable?

fluffy: this applies to any language w/ dictionary inheritance

ekr: no opinion about dictionaries permitting this behavior
... it's important to be able to add additional properties
... if that can't be made to work, that's death for dictionary

jan: i think Dictionary doesn't have a sequence on purpose
... as a useful constraint
... you can't know if a caller had a meaning in their sequence or not
... a sequential dictionary would be a less restrictive type

fluffy: write in the spec that objects will inherit from the type defined in the spec

hta: sold

adambe: so, StatsReport is a map?
... and we need unique key for each object?

hta: key was unique in that object

juberti: top thing is map->RTCStatsObject

hta: the dictionary's become interface *RTP* : RTCStatsObject

martin__: RTCStatsReport is second level of ugly

hta: it's just a map

martin__: a map of id's to things
... but why do we need ids?
... why are they the key?
... why not rtcstatsobject.type ?
... given id's aren't stable across calls

adambe: at time X
... you get a report
... check stats for some subreport component
... at time Y
... you get a report
... you want to compare reports
... you need to know that stats report is comparable

"Comment 22"

martin__: indexing by type
... i need to go back to usage patterns

hta: by id gives you one
... by type could give you more
... an id could be an implementation defined string/random number

martin__: but that can be correlated to other stats object at other random times for the same stats object?

hta: it's the identifier of the underlying object's specific stats property

<dom> Harald stats v2 proposal

adambe: how do we define RTCStatsReport ?

martin__: I think WebIDL has operator [index]
... or getter()

hta: i can't say definitely w/o `JS the good parts`
... i've seen filter()
... if primary map is by id, then it's easy to get by type

adambe: statsreport is only by id
... people implement by type themselves

hta: i think my life is simpler

juberti: what's the timestamp unit?

hta: `long` is a mistake

adambe: i argued for DOMTimeStamp

hta: I argued for Date

ekr: i'd be sad if i couldn't express time in units smaller than ms
... requirement is sub-ms resolution

<dom> High Resolution Time

juberti: FP or us?

martin__: nothing native provides one

dom: High Resolution Time does this

Josh_Soref: i believe it's from Web Perf

hta: adopt High Res Time as soon as it goes to the list
... i implemented this as long and was surprised it didn't overflow

[ No more slides ]

juberti: on naming
... RTCStatsObject is the parent type
... derived types are RTPStream
... do we want `Stats` in the names?
... i could see having an RTPStream object at a future time
... i think they should show that they're Stats

adambe: how do we keep track of remote stats?
... are they properties of a local stat
... or another property?

hta: most of the time, properties
... i defined RTPStream `other end is`
... one comes from RTCP and one's local
... linking objects together is a special case

adambe: my preference was to target the remote object directly

hta: the weird point is
... if you get partial stats
... sometimes the other side falls outside your filter

adambe: but you only filter on one level
... remote shouldn't be null or

[ scribe is getting lost ]

hta: if you filter, you get n+object graph
... you could have stats for other tracks
... one case you look up id and it's there
... and one case the id isn't there
... if there's a point about filtering

fluffy: i could live with this
... i'd prefer something more
... trying to treat remote stats slightly differently doesn't make sense to me
... i'd suggest having 4 sets of statistics ({Local,Remote}{Incoming,Outgoing}RTPStreamStats)
... and then stuff all 4 in one object

hta: if you group, you impose a tree structure
... i have peerconnection -> streams -> tracks
... i have transports -> tracks
... not the same mapping

"relations are good, normal forms are good, trees are bad"

juberti: on name
... type
... do we prefer to spell out type
... as opposed to instanceof
... you have a bunch of RTCStats objects that are more derived
... you can look at the parent name property
... should we use instanceof?

Josh_Soref: if you want to duct-type an object, instanceof doesn't work

martin__: i wrote up an example of how to use this
... it's clumsy if i'm looking for a particular type
... how to figure out which type i'm looking for

hta: leaf hierarchy
... reuse inheritance for more attributes
... type field isn't useful anymore
... is it incoming, or googlespecial or ...
... so we need type documentation

adambe: some way to tell getstats what you want to poll for?
... it'd be easier if you could somehow know what to expect

stefanh: Lunch

[ +1 +1 +1 ]

fluffy: is there uncertainty to the ietf-stuff
... for what to do?

hta: for stats, adambe needs to edit as agreed
... adambe and i will have a telco next week

fluffy: one of you will create a document-ietf mirror of burn

hta: which registry?

fluffy: do we create a registry of stats?

hta: i think we should

fluffy: expert review?

TedHardie: there's a W3C side process for creating registries

adambe: what about additional ones
... do we want some mandatory stats?

hta: i think the spec is unreadable w/o some examples
... the document should register those

[ Lunch ]

[ Meeting resumes as IETF after Lunch ]

TedHardie: we'll try to close IETF on-time if not early due to SnowPocalypse
... i know we officially scheduled 5
... i'd suggest we target resuming @ 1:15