See also: IRC log
<trackbot> Date: 07 February 2013
<dom> stefanh: propose to adapt agenda in light of lack of progress on SDP interface yesterday
<dom> ... instead of SDP agenda item, we'll look at video scaling and cropping in Media Capture Task Force
<scribe> scribe: timeless
[ Conclusions Media Cap discussions Boston ]
[ hta reads slide ]
hta: any comments/complaints?
[ None ]
cullen: thanks guys for doing this
[ MediaStream Cropping ]
justin: this is a quick set of
slides i put together
... for how a camera could stream could be provided in an
aspect ratio not equal to native
[ Problem statement ]
<dom> Justin's slides on aspect ratio
justin: most sensors and cameras
are moving to 16:9 capture
... not all cameras are new
... quite a few are 4:3
... it's hard for a full screen gui, especially full screen gui
@ 16:9
... <video/> will always pad to fit
... as opposed to crop to fit
... even if <video/> would crop to fit
... you wouldn't want to encode those bits that'd be cropped at
the other end
... if you're doing local preview, you'd probably want to see
the other side
... so people can see if they're out of from
[ Solution: crop 4:3 to 16:9 ]
justin: discard top/bottom (or
left/right if needed)
... Flash's camera api would do this if you asked for an
unsupported res
... instangram likes squares, so you could get this
... i'm only concerned w/ crop - letterboxing/pillarboxing are
available from <video/>
[ General Approach ]
justin: opt in, new mandatory
constraint
... you want this specific height, width
... width.max,min=640
... height.max,min=360
... allowCrop=true
... if camera supports resolution that falls within mandatory
constraints, use that res
... if camera supports res that exceeds mandatory constraints,
allowCrop=true to crop satisfy mandatory constraints
dom: if this is mandatory
... how does it fit?
... it doesn't seem like a `selection` critera
justin: it's more for having
chosen a camera
... selecting settings later
dom: so, it's a constraint that may be more for control than selection
justin: i'm intending it for that, yes
burn: this is control, not
selection
... why mandatory { allowCrop }
... instead of mandatory/optional { cropIfNeeded }
justin: for optional things
<dom> [this seems to point to a growing divergence between constraints-for-selection and constraints-for-setting]
justin: if these bounds are not
hard bounds
... then it'd be a closest match
burn: if it didn't get it, it
didn't get it
... the point of optional constraints is notifying them if they
can't be satisfied
... i'm proposing `cropIfNeeded`
... if you put it in mandatory, then it would crop if it needs
to and can
justin: do you have an
example?
... i have a camera that can or can't crop in hardware
burn: you might have other
optional constraints relating to aspectRatio
... if you get those, you might want cropping,
... if not, you might not care
JonLennox: comment, that might
make this harder
... i'm often in a situation of VGA camera
... cropping `middle 2/3` makes it easier
... but on a mobile phone in landscape, cropping like that
doesn't do what i want
justin: i have another
proposal
... PeterThatcher mentioned that problem on the ride over
fluffy: trying to simplify
things
... i assumed crop was always true
... if they asked for 16:9 and the input is 9:16, they get a
postage stamp
... person rotates phone, problem goes away
... caution that adding complexity won't help people
... what happens if it's mandatory and it comes back
`can't`
hta: i kind of like the general
approach
... but i dislike the examples
... i live in the world of resizable windows and interchangable
cameras
... i want to write a UI once
... that will work when the user resizes the window
... and works w/ Barbie Doll camera
... and an HD camera
... i want 1 of 2 things
... either everything in picture field
... or i want picture to fill my field
... in neither situation should we stretch the picture
... remove the talk about min/max, and use the res.
... if you need to crop, crop
justin: are you +1'ing fluffy
?
... if you ask for a res, and
hta: when media is flowing
... browser knows where it's going
... and it knows where it's coming from
... it will have to do a fit operation
... this is an instruction to browser to crop instead of
letterbox
... it doesn't instruct browser to aim for res
timeless: he's complaining about the example
justin: he's saying constraint pillarbox/crop
hta: my real window size is
250:172
... it should fit to that
... the best way to achieve that
... i think the browser should do what it needs to do
stefanh: with <video/> it
can adjust to video aspectRatio
... and there are examples of using <canvas/> to
crop
... canvas uses lots of power
... and you lose advantage of sending non-displayed bits
justin: i was going to add
<canvas/> in an earlier slide
... if we had a Media Stream Processing API
[ laughter ]
justin: doing things in the
GPU
... could be very efficient
derf: i want to apply this to any
video stream
... not just cameras
... according to settings, a <video/> can't have this
applied
[ Examples ]
justin: this isn't the alternate
proposal
... just how to use this constraint
... 720p sensor, want 4:3
width.max,min=960
height.max,min=720
allowCrop=true
burn: on this topic of do we let
browser do its wisest thing
... or let JS dev give a preference
... i don't think it makes it more complex to allow both
cases
... it's quite productive to let the user choice
... crop, pad, best-guess
justin: that's what i heard fluffy say
burn: if there's a mismatch
between <track> request and destination
... then you can express a preference as an app writer
... for cropping, or padding, or not caring
justin: how do you recognize this mismatch when the destination is PeerConnection
timeless: shouldn't PeerConnection be able to express a resolution?
burn: there's no way to express
PeerConnection res as you do for <video/>
... unless we want to add that
[ Yes ]
burn: that would address a lot of
this
... because then it becomes parallel to <video/>
fluffy: what i heard by hta was
what i said
... a resize by Pretty, Ugly, ...
... for something that crosses the PeerConnection
... if we want that to propagate back to the origin
... the resize will have to go back across the
PeerConnection
... involving a renegotiation
... complicated, but very nice
... if no one sets constraints
... but we need a way to say "i'm going to send 640x480"
... by creating a track with that
... and PeerConnection says "oh, that's what i'm sending"
... and then it doesn't negotiate
justin: someone changes window
size
... and that pushes it all the way back
... if someone changes window size and camera has to stop and
restart,
... that yields wacky behavior
fluffy: if camera is SIF
camera
... and someone selects QCIF
... allow crop also
... have constraints?
justin: proposing
<dom> s/QSIF/QCIF
justin: PeerConnection encoder can do cropping internally
<dom> CIF on wikipedia
justin: the point is to give
something to the encoder to do efficient scales
... for power of two scales
... for efficient matches of what output can display
martin__: the more i think of
this
... the more i think i have too many ways to do the same
thing
... this seems to belong in the renderer
... we have <video/> it may
pillarbox/letterbox/crop
... if we treat PeerConnection as renderer
... then i don't think we need constraints
justin: assume a single
camera
... no choosing
... for WYSIWYG
... do we have 2 mechanisms or 1 ?
... same way to ask camera for get X res, same way for X'
res
martin__: two ways
... @ <video/>
... and a way @ Track()
... which may or may not
... if each sink is the only place to set these
properties
... the only constraint would be for selection
... i'm coming to the conclusion that that's pointless
... since in most cases it's orientation of camera that matters
most
hta: device selection is different topic
ekr: whatever camera
aspects
... if i plug in arbitrary res
... to <video>
... we expect something as reasonable as possible
[ it will rescale in letterbox or pillarbox ]
martin__: there's no way to control it
justin: is it specified?
stefanh: i think it's
specified
... you can ask it to auto grow
justin: you can never crop
ekr: anything really
sophisticated
... like Media Stream Processing API
... want something analog to <video/>
... i want to force fit this camera to this stream
... this is a stop-gap?
justin: fixing a common problem w/ an efficient+cheap solution
ekr: i could live w/ either
@source or @sink
... i think i exploded at pushing across Connection
stefanh: to martin__
<dom> the aspect ratio of the video, the video will be shown letterboxed or pillarboxed. Areas of the element's playback area that do not contain the video represent nothing."
stefanh: to add Crop <media> element?
martin__: yeah
burn: i think simplest logical
model for all of this
... is your Track is used to identify what's natively
produced
... and Sink is what you expect to get out
... and influences where you munge
... so PeerConnection should have things like
<video>
... i think constraints on Track still make sense
... so you can ask for High Res from Camera (which offers High
Res and Low Res)
... so an app writer can say "these matter to me"
... so sink does processing
... and make it clear that's where it is
justin: so that's a proposal to
flag it
... at ...
burn: allow back propagation or
not is up to us
... we could allow settings on PeerConnection to allow settings
to go across
... PeerConnection gets to decide how its properties as a sink
are set
justin: choosing how a device is
opened up
... i argue this is analogous
... saying "give me X", or saying "give me X'"
... we could talk about feedback mechanism
... saying `camera can produce X res's`
... the reason allowCrop=true is there
... is for if you're in Portrait mode
... and you don't want to do cropping
... maybe apps don't ever do that
... give me these pixels no matter what
... give apps some flexibility
burn: i'm not recommending taking
away control from app writer
... question is where to do it
... on Sink or on Track
PeterThatcher: is it safe to say
we want to avoid reopening camera?
... does app tell browser this?
justin: most flexible is to open
@high res
... and crop/scale
... cameras have non-0 reset time
... and there's noticable artifact as they stop/restart
camera
PeterThatcher: would support controlling the crop at camera open time to avoid other constraints changing and causing re-open
<dom> ScribeNick: dom
justin: changing camera after feedback from sink
martin__: the simple response to
that is: don't do that
... if you're operating in the wrong mode for the display
you're opearting in, you'll have to deal with it at some
point
... This proposal is adding processing in the pipeline, which
is what I wanted to avoid
... leaving constraints for elements that matter for the
hardware
... anything related to processing should be left up to a
processing api
... that api would provide a way to crop the video among other
things
justin: that would be great, but that expands the scope of needed work
martin__: we would need to gather use cases, but that would be much cleaner than trying to push processing through constraints
adambe: everytime we talk about
constraints, it sounds like this could be for sources, tracks,
or sinks
... that is very confusing
... it would be easier if only of these should be the focus of
constraints
burn: I have a similar comment; I think it should apply to two, not three
adambe: we have the recorder as a
sink, for peerconnection, stefanh proposed a transport handler
to support setting priority etc
... for the MediaElement, you can control the width and height,
but I don't know how you would control framerate, etc
... It seems more natural to let the sink in control
Dan_Druta: I heard two aspects of
applying constraints for: efficiency, and user experience
... we should be very clear about it when we talk about pros
and cons
juberti: re the sink driving, that's an important model, but I don't think we should force everyone under that model
gmandyam: in the media capture
call in December, we discussed @@@
... you can set options for recording independently of the
source
... it's not a real-time operation either, so with different
considerations than peerconnection
juberti: setting stuff at the source level doesn't invalidate options at the recording level
burn: the current state of the
settings proposal that we talked about, and the world we live
in the current spec
... is that of source, tracks, and sinks, you can only set
constraints on tracks and sinks
... peerconnection doesn't have the same kinf of settings
capabilities
... there is a need to control what the source provides
natively
... (if the device cheats e.g. with Apple, there is nothing we
can do about it)
... we only two, but we have three places
... we could set constraints on sources, tracks, sinks
... the problem with constraints on peerconnection, you can
have multiple tracks going in a peerconnection
... each of these tracks could have their own
configuration
... this is why we've been talking about control on
tracks
... the problem is to determine whether munging is done at the
track abstraction level (and the source is where you set what
you want), or at the sink abstraction level (and the track is
where you set what you want)
fluffy: +1 to burn
... we're lacking a mental model of all this
... there are sources, sinks, and tracks connect them
together
... the problem with sinks is that we don't have much control
about it
<gmandyam> @@@ = At December MediaCap TF call, the participants agreed that width-height combo settings are required for the recording API. I proposed a possible setting to the mailing list.
fluffy: as a result, tracks are
preferred location for exercising control
... if you put constraint on tracks, that affect what's
produced by that track
... for the sources, the general model is that they look at all
the tracks they deliver, and they select the best setting that
allow them to satisfy the tracks they're producing
... As a result, the mental model should be that Tracks are
where we want to set constraints preferably
juberti: that makes sense to me
Josh_Soref: regarding the risk of
latency attached to reopening a camera
... could this be something we can prevent using a
constraint?
... (preventing from reopening a camera to adjust to a new
setting)
... On the PeerConnection bit, I've always assumed @@@
stefanh: I think we need more
control over transmission media
... we could apply that to tracks, or provide an api in
peerconnection per-track
juberti: I would prefer to apply it via peerconnection
stefanh: I agree
hta: the video element has the CSS3 property to define the cropping/fitting model
martin__: I don't mind fluffy's
suggestions, except that it means you now have two ways to
control this for the video element
... I'm kind of tempted that we don't have this setting API for
that
... I'll have to talk with Travis about that
juberti: I have an alternate proposal
<hta> (the css3 property is called object-fit - it took me a while to find it)
juberti: where you define the maximal crop aspect ration
-> http://dev.w3.org/csswg/css3-images/#object-fit object-fit in CSS Image Values and Replaced Content Module Level 3
Josh_Soref: in the recorder case,
there isn't necessary a post-processing server somewhere
else
... I can be doing local recording
juberti: the cropAspectRatio is the process aspect ratio
@@@: we would make it clear that process stuff would only be used for peerconnection
cullen: do you have a slide on scaling?
juberti: no
cullen: I would love that we say we never scale up
martin__: disagree, it has to
happen in some cases
... I think we're talking about processing, and where that
processing is applied
stefanh: to avoid upscaling, we would have to change the mediaelement, since it scales up
cullen: I was referring to not allowing to scale up at the track level, but I think the length of the line shows I'm losing on that one
ekr: what does it mean to scale up a track?
cullen: say you've acquired a sd
camera
... and then you apply a constraint to the track to get a
bigger resolution
ekr: so I have a SD source that
gets a SD track, plugging it into an HD video tag — that will
be ugly
... if you were to apply to same setting to a peerconnection,
the same thing would happen
... what you're saying that we should not allow to augment the
resolution of a native track
burn: I think you should have
control
... but fluffy, you're badly schizophrenic
... if the constraint you set on the track is the output of the
track
... and it's ok for munging to happen
... then scaling up has to be allowed to happen
... what we would want is to have constraints to limit the way
the processing can be done
... and I think that's what justin is describing
juberti: the goal is to provide a
simpler way to do processing without providing a full-blown
graph processing api
... I worry about an authoritarian model where @@@
... I don't think it makes sense to send the quadruple amount
of bits with no increased quality
tim: afaict, no browser has implemented object-fit property except opera in the past two years
-> http://caniuse.com/object-fit object-fit in caniuse
jan: we're a bit off-track with
upscaling
... we've talked about source constraints and track
constraints
... if we had peerconnection.maxWidth,
peerConnection.maxHeight, this would cutshort this whole
debate
juberti: I think the
peerconnection should have specific control on what each track
aspect is
... I think we're pretty far afield to specific problem I'm
trying to solve here
martin__: if you're concerned
about sending more bits over a PeerConnection than is
necessary,
... having these settings on PeerConnection allows the PC to
say "you've set this higher than what the source provides"
Jan: I would modify my earlier
proposal to have PeerConnection say the aspect ration of what
the client will care about (e.g. 4x3)
... this is only for peerconnection
juberti: not really, since
object-fit is not implemented
... some cameras can do cropping at the hardware level, but not
all of them
... this would let us solve the problem to these various
cases
... it's not the general mediaprocessing api, but it has a
better RoI
hta: I think there is some level
agreement that cropping needs to be able happen
... and the application should be able to have some control
over it
... there is a fair bit of debate on what the model is on what
constraints apply to, and how they show up at the destination
of the track
... vs modification at the source of the track
... We kind of passed that boundary when we agreed to be able
to produce multiple resolutions from a single source
... I would like to ask martin to write up his concerns on
setting constraints on track
... and propose an alternate approach
<hta> ACTION: Martin to write up his concerns with using constraints on track to manipulate stuff that might apply to sources and might apply to sinks [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action01]
<trackbot> Created ACTION-83 - Write up his concerns with using constraints on track to manipulate stuff that might apply to sources and might apply to sinks [on Martin Thomson - due 2013-02-14].
hta: I would like justin to take an action item to describe this constraint in a way that can be included in the document
<scribe> ACTION: Justin to describe the constraint for aspect ratio for potential inclusion in the document [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action02]
<trackbot> Created ACTION-84 - Describe the constraint for aspect ratio for potential inclusion in the document [on Justin Uberti - due 2013-02-14].
<timeless> scribe: timeless
hta: there are cases where we
need to provide more information than just the error name
... e.g. when there is an sdp error
... we want the line of where the SDP arose
<inserted> ScribeNick: timeless
hta: could people help me refresh my memory?
burn: we were trying to finalize
the principles
... but we decided we needed to look at specific
... and Adam Roach said we needed to hear from
implementers
... as they implement
... instead of "ooh, maybe it will do this, maybe it will do
that"
ekr: what errors we're
seeing
... what you need to return
... how often, and when
... justin and i are debating Error1, or Error2 and
Error3
... i mostly see malformed SDP
... a browser screwed it up, or someone mangled it
... a parser of why i didn't like it and what it was
... Adam Roach pointed out the same need for ICE
... when i look at existing ICE information
... that would be a long way toward knowing what's going
on
... a way to detail every check
... would be useful
... very confusing errors
... the only thing in the docs at all is detailed ICE info
justin: for Chrome 25, we'll have
which line of SDP it choked on
... SDP people complain about Dom11 for not having crypto
lines
... we should see if that's useful
... we also created chrome:ice-debugging (not for JS)
... to allow people to debug apps w/o exposing js apis to
apps
... we'll have more debugging tools
... before we invent too many more things
ekr: most of these errors are
programming errors
... i think it's a great idea to have inspectors that are only
accessible to programmer
<dom> [The WebSockets API is very careful in not exposing too many details about errors: http://www.w3.org/TR/websockets/#feedback-from-the-protocol ; not sure if there are similar risks in our case]
ekr: you test when you write
code
... you don't know why
... once you get it working, it works
... ICE check failed on component 1 to the user
... is unlikely to help the user
fluffy: we're putting your email
address in our errors
... i don't think we've reviewed the doc for errors that can
come up
... we're seeing implementations
... a port of Firefox that also does 264
... not being able to report up mismatches
... problems that you don't see since you only do one codec
derf: how we report errors is
good
... we need to define what happens when there's an error
... you might parse SDP, it might be valid, but not apply to
session state
... mismatched m=lines between offer and answer
... i don't care about behavior, but it needs to be
consistent
... we could Tear down the entire call
... we could not access the entire SDP
... or browser crashes
... but it shouldn't be inconsistent between browsers
dom: WebSockets API is
careful
... about avoiding exposing details
ekr: SDP exposes the entire Network Topology
<dom> [so should we drop SDP? >:]
ekr: hta's proposal of
statistics
... could be useful
... or if i'm showing my video instead of remote
... if there's no RTP coming in
... re derf's proposal
... there's going to be a whole class of things
... passing bogus SDP
... sometimes that's a parse error
... sometimes that's an application error
... standards should encourage right behavior
... but if you get halfway through applying an SDP
... i can't roll-back
... it might be better for me to close
juberti: someone had a action to
describe how rollback should happen
... two sets of errors
... @SDP layer
... either not eat
... or tear down
... or @SRTP
... and we could blow up in Security layer
... we can make best practices
hta: implementer feedback
... some parse errors
... are easy, some are harder to pin down
... the missing crypto line is hard
... which line is the one to report it for?
... it would be nice to have some descriptive text
... not for applications
... but useful for debuggers
<dom> [oh boy, are we now describing an SDP parser API?]
hta: or maybe useful advice
ekr: missing crypto line v.
missing fingerprint
... having looked at jingle
... it wouldn't be at parse time
... it'd be later
... you need a textual report
... "i'm sad for reason: ..."
... parsers print to logging but don't propagate
juberti: that landed last
night
... in Canary
stefanh: no big need for adding
errors
... but more on what happens when you encounter an error
hta: this is part of procedure
description for setLocalDescription/...
... we left off precise language
... on the idea of getting the general principles before we
write precise language
... i think it's the time to write precise language
[ Coffee Break for 15 minutes ]
jesup: W3 flavor of data channel
discussion
... i hope less controversial than the IETF flavor
... maybe we'll make a decision
[ DataChannel JS API ]
jesup: feedback from Mozilla Devs
working on those apis
... has been generally positive
... it's worked the way they've expected
... primary unresolved area
... is how to create data channel and how it interacts w/
createOffer/createAnswer
[ Changes to API ]
jesup: IETF 85 ... protocol to add data channels to agree on interop
martin__: the web socket idea of a sub protocol?
jesup: i guess so
... an IANA registered string
... if you do this, you agree you're exchanging in some
form
... since we're mirroring WebSocket
... this is that thing from WebSocket
adambe: latest version of WebSocket lets you say you accept multiple protocols?
jesup: but you can only use one once it's in flight
<dom> WebSocket Subprotocols registry
jesup: our way to create data channel is different from WebSocket
martin__: no need to
negotiate
... it's with itself
jesup: you would need a way to signal back agreement, complexity for no reason
[ Proposal ]
jesup: like yesterday, but w/o
SDP
... all the JS layer
... no DataChannel unless you call pc.createDataChannel()
... you can call it any number of times before
createOffer/createAnswer
... if called after initial createOffer, it triggers a
negotiationneeded
hta: createOffer or setLocalDescription?
jesup: before SDP or after SDP is
the distinction
... relative ICE things
hta: what if they createOffer throw it away and call createOffer again?
jesup: maybe `initial` is wrong
hta: one solution is first call triggers negotiationneeded
jesup: createStream throw away
stream, stream is still there
... same for data channels
martin__: seems like there's some need for ...
derf: we had an AI for when negotiationneeded fires
martin__: and when other things fire after
[ Problems ]
jesup: unless you sniff SDP, you won't know if there was a data channel
<martin__> the question is a) when onnegotiationneeded fires and b) when it clears such that it could fire again
jesup: we can't add an m=line in
an answer
... if you want it, you'd need a second round of
offer-answer
[ Example ]
<derf> martin__, I suspect the answer to b) is that it can always fire again.
[ Decisions! (Please!) ]
jesup: use createDataChannel to
trigger m=application line
... if needed, createDataChannel will call
negotiationneeded
martin__: need to be >0 open
channels to include m=application
... because you could create a data channel, close it, and then
createOffer
hta: i'm fine w/ this
... is that what you want?
jesup: basically
XXX
paul: namespace for sub
protocols?
... subordinate to webrtc channel namespace?
... should be same namespace as websocket subprotocols?
... i don't know that
<martin__> ACTION jesup to if we replace onnegotiationneeded conditions to the same as those we have for addStream()
<trackbot> Error finding 'jesup'. You can review and register nicknames at <http://www.w3.org/2011/04/webrtc/track/users>.
jesup: i think it's an IANA register
paul: just want a procedure
<martin__> ACTION Randell to make onnegotiationneeded conditions the same as those we have for addStream()
<trackbot> Error finding 'Randell'. You can review and register nicknames at <http://www.w3.org/2011/04/webrtc/track/users>.
paul: to define these
TedHardie: it's a first come
first server registry
... set up by WebSocket folks
paul: single level name
hta: you can define protocols for
data channel
... that don't work well w/ web sockets
... that might be an argument for different registry
jesup: not an argument for different registry, just don't use w/ websocket
TedHardie: for this room, push to IETF side
jesup: i agree
... what `protocol` means shall be defined by IETF
<martin__> OK, randell isn't a registered user
Dan_Druta: protocol isn't limited to registry, right?
jesup: if it wishes to ignore the
registry, nothing can force it
... I yield the remainder of my time
stefanh: we made a decision to go w/ this w/ hta's amendment
<dom> ACTION: stefanh to get Randell to make onnegotiationneeded conditions the same as those we have for addStream( [recorded in http://www.w3.org/2013/02/07-webrtc-minutes.html#action03]
<trackbot> Created ACTION-85 - Get Randell to make onnegotiationneeded conditions the same as those we have for addStream( [on Stefan Håkansson - due 2013-02-14].
[ Stats ]
[ hta defines Microphones `chair at desk`, `speaker at wireless`, `individual at center mic` ]
<scribe> [ New API ]
hta: changing datatype of what
comes back on stats report
... to be an object that's able to tell you the
identifiers
... and able to get those objects
... each object has type and id
... and set of names, and a getter to get value for name
juberti: like a Dictionary?
<scribe> [ New Objects ]
hta: exactly
... Dictionaries are a name value mapping
... I decided this is a fine way of putting stuff into
documentation
... that can be checked with a WebIDL parser
... sequence returned order is somewhat curious
... depends on alphabetical names
... and based on hierarchy
... things defined in derived dictionary are added to the
sequence (sorted) after the original dictionary's list
... this is a pain in the posterior
... i looked into the state of a Dictionary
... and how to return one from a function
... the answer i got is that `it's doable, w/ 3 experts in DOM
state manipulation`
... so i propose no dictionaries
... [as from the previous slide ]
[ Principles of object definition ]
hta: use Dictionary as
documentation method
... pointers are done via ID
... Keep it simple (and well defined)
martin__: i'm unsympathetic to the hard to implement
hta: did you implement something that fully implements dictionary
martin__: one thing i didn't deal
w/ was extra params
... the idea that you'd have trouble passing objects to js as
hard to do...
... we've discussed this at length
... the things we're getting are key-value-pairs w/ primitives
and maybe arrays
... but you're forcing them to use unfamiliar constructs
adambe: JS has a way to deal w/
KeyValue pairs
... couldn't we say we're returning a JS object w/ these
values
... and not promise order
... that gives us enumeration and lookup from JS
hta: one vote for
Dictionary
... and one comment for Object
fluffy: in favor of making this work
<dom> [I think you can use [NoInterfaceObject] interface to describe regular JavaScript objects]
fluffy: keen to restrict
semantics to that of Dictionary
... if function calls get mapped to thing, this would be a
disaster
... nearly every language has something like Dictionary
... I want Dictionary semantics, ... ASN.1 - i don't care
ekr: is there an argument against
these being properties
... that isn't rooted in implementation details
... why isn't names() and getValue() just properties of the
object
... one is that it's Lousy for JS user
... and one that it's inconvenient for the C++ WebKit dev
... is it just PITA for WebKit?
... i'd have expected that a param was property index
(.keys)
... and reference by .<> or [<>]
fluffy: you expect Dictionary
ekr: what's the merits of this other than implementation?
juberti: if it was []/. instead of getValue()
dom: i understand Dictionary is a
pain
... but why not Interface
... this is similar to GeoLoc in a way
... they use [NoInterfaceObject]
... we shouldn't define our own way of declaring our own local
data
... WebIDL w/ interfaces provides that
martin__: use partial interfaces
hta: partial interfaces w/o source
juberti: why objects
... couldn't we make them more like regular dictionaries?
hta: if `void (RTCStatsReport
statsReport)` with a map
... Dictionary provides some guarantees and constraints i don't
want
martin__: it doesn't need to be a
Dictionary
... it could be [] or .
... you can do this in JS in browsers w/ handlers
... i don't see a problem w/ implementation
... if you aren't backing directly w/ C++
... we need a map
adambe: `statsReport`
... report is a complete thing
... and you have subreports
... you want two levels
... you could guarantee a timestamp, type, id
... why are those on a different level?
martin__: individual stats object could be created at different times
adambe: move timetamp/id into
getValue?
... i'm disregarding the top object
[ RTCStatsReport ]
adambe: i don't see why we need a stats.type v. stats.getValue(x).
hta: how do you specify that in WebIDL
martin__: i'd rather show example code for exact number of packets
dom: I think in WebIDL, your
additional data structure inherits from a core interface
... Object is a Map
fluffy: don't drive from
WebIDL
... figure out from how you want to use it
... i think it generally does
hta: i don't have slides w/
code
... if i have dictionary foo { int blah; }
... i can't have additional attributes
martin__: no one sees the
dictionary type
... so you can have dictionary privatefoo : foo {...}
hta: they are unobservable?
fluffy: this applies to any language w/ dictionary inheritance
ekr: no opinion about
dictionaries permitting this behavior
... it's important to be able to add additional
properties
... if that can't be made to work, that's death for
dictionary
jan: i think Dictionary doesn't
have a sequence on purpose
... as a useful constraint
... you can't know if a caller had a meaning in their sequence
or not
... a sequential dictionary would be a less restrictive
type
fluffy: write in the spec that objects will inherit from the type defined in the spec
hta: sold
adambe: so, StatsReport is a
map?
... and we need unique key for each object?
hta: key was unique in that object
juberti: top thing is map->RTCStatsObject
hta: the dictionary's become interface *RTP* : RTCStatsObject
martin__: RTCStatsReport is second level of ugly
hta: it's just a map
martin__: a map of id's to
things
... but why do we need ids?
... why are they the key?
... why not rtcstatsobject.type ?
... given id's aren't stable across calls
adambe: at time X
... you get a report
... check stats for some subreport component
... at time Y
... you get a report
... you want to compare reports
... you need to know that stats report is comparable
"Comment 22"
martin__: indexing by type
... i need to go back to usage patterns
hta: by id gives you one
... by type could give you more
... an id could be an implementation defined string/random
number
martin__: but that can be correlated to other stats object at other random times for the same stats object?
hta: it's the identifier of the underlying object's specific stats property
<dom> Harald stats v2 proposal
adambe: how do we define RTCStatsReport ?
martin__: I think WebIDL has
operator [index]
... or getter()
hta: i can't say definitely w/o
`JS the good parts`
... i've seen filter()
... if primary map is by id, then it's easy to get by type
adambe: statsreport is only by
id
... people implement by type themselves
hta: i think my life is simpler
juberti: what's the timestamp unit?
hta: `long` is a mistake
adambe: i argued for DOMTimeStamp
hta: I argued for Date
ekr: i'd be sad if i couldn't
express time in units smaller than ms
... requirement is sub-ms resolution
<dom> High Resolution Time
juberti: FP or us?
martin__: nothing native provides one
dom: High Resolution Time does this
Josh_Soref: i believe it's from Web Perf
hta: adopt High Res Time as soon
as it goes to the list
... i implemented this as long and was surprised it didn't
overflow
[ No more slides ]
juberti: on naming
... RTCStatsObject is the parent type
... derived types are RTPStream
... do we want `Stats` in the names?
... i could see having an RTPStream object at a future
time
... i think they should show that they're Stats
adambe: how do we keep track of
remote stats?
... are they properties of a local stat
... or another property?
hta: most of the time,
properties
... i defined RTPStream `other end is`
... one comes from RTCP and one's local
... linking objects together is a special case
adambe: my preference was to target the remote object directly
hta: the weird point is
... if you get partial stats
... sometimes the other side falls outside your filter
adambe: but you only filter on
one level
... remote shouldn't be null or
[ scribe is getting lost ]
hta: if you filter, you get
n+object graph
... you could have stats for other tracks
... one case you look up id and it's there
... and one case the id isn't there
... if there's a point about filtering
fluffy: i could live with
this
... i'd prefer something more
... trying to treat remote stats slightly differently doesn't
make sense to me
... i'd suggest having 4 sets of statistics
({Local,Remote}{Incoming,Outgoing}RTPStreamStats)
... and then stuff all 4 in one object
hta: if you group, you impose a
tree structure
... i have peerconnection -> streams -> tracks
... i have transports -> tracks
... not the same mapping
"relations are good, normal forms are good, trees are bad"
juberti: on name
... type
... do we prefer to spell out type
... as opposed to instanceof
... you have a bunch of RTCStats objects that are more
derived
... you can look at the parent name property
... should we use instanceof?
Josh_Soref: if you want to duct-type an object, instanceof doesn't work
martin__: i wrote up an example
of how to use this
... it's clumsy if i'm looking for a particular type
... how to figure out which type i'm looking for
hta: leaf hierarchy
... reuse inheritance for more attributes
... type field isn't useful anymore
... is it incoming, or googlespecial or ...
... so we need type documentation
adambe: some way to tell getstats
what you want to poll for?
... it'd be easier if you could somehow know what to expect
stefanh: Lunch
[ +1 +1 +1 ]
fluffy: is there uncertainty to
the ietf-stuff
... for what to do?
hta: for stats, adambe needs to
edit as agreed
... adambe and i will have a telco next week
fluffy: one of you will create a document-ietf mirror of burn
hta: which registry?
fluffy: do we create a registry of stats?
hta: i think we should
fluffy: expert review?
TedHardie: there's a W3C side process for creating registries
adambe: what about additional
ones
... do we want some mandatory stats?
hta: i think the spec is
unreadable w/o some examples
... the document should register those
[ Lunch ]
[ Meeting resumes as IETF after Lunch ]
TedHardie: we'll try to close
IETF on-time if not early due to SnowPocalypse
... i know we officially scheduled 5
... i'd suggest we target resuming @ 1:15