Media Capture Task Force F2F Meeting

<trackbot> Date: 06 February 2013

<dom> ScribeNick: gmandyam

hta: going over conclusions from Feb. 5
... Device enumeration has been concluded to be necessary, but there are privacy issues.

Martin: privacy issues were not in scope of current proposal, but for suggested expanded capabilities. Additional device info could potentially improve privacy.

hta: Error handling will not change from current model.

Cullen: we should be consisten as to how we do error handling

hta: "Immediate Stream" ,i.e. synchronous gUM. This belongs in a sense to PeerConnection.

<dom> Media Streams and Identity slides

[ General Idea: Grant Media Access to an Idenity ]

[ Topics ]

[ Stream Isolation ]

<timeless> Scribe: Josh_Soref

<timeless> scribenick: timeless

[ History ]

ekr: we got three proposals
... I presented a 1st draft in Lyon
... i never got around to writing it up
... this is a detailed description of my proposal

[ Three ways to call gUM (all constraints) ]

ekr: * Call gUM()
... * Call gUM(noaccess=true)
... I won't touch the stream
... * Call gUM(peerIdentity=bob@example.com)
... This stream will only be sent to a certain identity

[ Call gUM() as normal ]

ekr: current behavior unchanged

<gmandyam> Hey Josh, are you scribing? I'll quit if you are. - Giri

[ noaccess=true ]

ekr: Media permissions checks as usual
... if site gets access, it gets permissions

<gmandyam> OK - let me know when you need a break

ekr: stream is isolated
... keying must be via DTLS
... you [martin__] will say no to identity assertion

martin__: this will enable creation of identity assertion based on stream

ekr: once peer connection is established,
... it either knows peer identity
... browser displays identity and "no access" indicator
... WebEx etc want to guarantee to others
... and customers want to check
... that the site didn't screw them over

[ Flow slide ]

ekr: indicator has an identity assertion

[ peerIdentity=bob@example.com ]

ekr: originally suggested by cullen?
... more enhanced from user perspective
... people seem to be concerned about allowing a site to have access for perpetuity
... "i only want to allow site to call X and Y and no one else"
... site indicates to user who the content will be sent to
... either RFC822 style email address
... or Browser if it has Contacts accesss, it could show name + icon

TedHardie: for Poker site example
... would it be skiptracer@pokersite.example ?
... but could you do player1@ ?
... or would you need a different mechanism

ekr: i think you'd outsource identity assertion

martin__: to do a Player 1
... you need player1@pokerstars.example
... the name is in this form
... and the domain for this domain can do the identity assertion

ekr: you need to be able to mechanically identify this person

TedHardie: this isn't valid for the poker table case without pokerstars preminting

martin__: just minting

Dan_Druta: there's a concern about information harvesting

gmandyam: what if you want to do a native media player
... if i want to use the recording api

martin__: you'd use the gUM(noaccess=false)
... you don't need to set the argument

gmandyam: what would this mean for access?

ekr: access will be on `origin` basis

gmandyam: for access to raw media?

ekr: you'll still have that
... this is a new way to restrict access

gmandyam: what motivates a web site to do this?

cullen: sites like WebEx want to be able to assure customers that they *don't* have access to the media
... they'll choose to use this mechanism
... and users could verify this by something morally equivalent to the lock icon today

justin: what does it mean for WebEx to not have access to the media?
... surely they'll have the data

cullen: WebEx won't have the keying material
... for a Point to Point Call
... it knows martin__ called ekr
... and for a TURN server, it might all flow through
... but it doesn't have the keying material
... for a 10 person conference
... you might know who is sending the content v. spreading
... but you won't know what was sent
... for WebEx or Google Hangouts
... if Microsoft and Silverlake
... are doing some deal about how Cisco would go out of business
... they want assurance that we aren't listening

justin: but in a WebEx conference
... the central node won't have any access to the media?
... what assertion covers the end to end
... that it won't have access to the media?

cullen: for 1-1 i'm saying we can do it

aroach: if identity asserter and site you're on
... are the same or are in collusion, you're ...

ekr: you're hosed
... this is about a site being able to provide you with evidence
... but you have to trust the source of the communicating counterpart

PeterThatcher: we're pushing trust from site hosting JS
... to site hosting identification

ekr: or a site with extra ...

PeterThatcher: are we confusing the user
... about do you trust X v. Y.com ?

ekr: always some risk
... people have some sense that domains control identities
... some sense,
... companies reassign email addresses
... Gmail can redirect email somewhere else
... 2. there's a lot of work in distributed identification systems
... BrowserID
... it's a lot more straightforward to do a calling site
... than an identity provider
... so your identity is likely to be by someone you trust
... but the caller site something less trusted
... 3. this is what we know how to do
... MediaStream can only be connected to a PeerConnection with identity provided
... completion would choke if call goes elsewhere

[ ekr explains call flow ]

ekr: that's the entirety of the mechanism
... i can send to the list on the details
... but it's simple

gmandyam: can you go back to the call flow slide?
... this is entirely voluntary
... the web site can contact the ID provider directly
... the web site can put up its own dialog box
... it can ask user "can i send to bob@example.com"
... why add to gUM

ekr: this is for Identity where you don't trust calling site
... this is about denying calling site access to media

gmandyam: the web site can get access to the media by not doing this

ekr: this is a way to build a site where the user can verify the site isn't sniffing

cullen: there's a reason the browser needs to show the Lock icon
... instead of the web site showing "secured by Thawt"

ekr: they do that to though ;-)

cullen: this is about the chrome of the browser showing the assertion

ami: how does bob tell his PeerConnection that he's bob@example.com

ekr: that's defined in IETF drafts

jesup: in addition to who puts up the assertion
... and whether they can access the media
... even if you trust the site to not access the media
... the site can always be hacked
... so trusting them
... if they're hacked
... worse case they won't get the media

ekr: i'll send detailed text for the proposal

Recording Consent

ekr: W3C (will have?) published recording API as FPWD
... do we need special recording consent?

[ Basic Recording Concept (review) ]

ekr: i think i'm summarizing correctly
... MediaStream
... feed it to MediaRecording() constructor
... js gets direct access to media

[ Why the special permissions? ]

ekr: quoting Ben Pedtick
... there are regulatory requirements for such notifications

[ Not the only way to get direct access ]

ekr: people suggested building it into the browser interface
... Pipe MediaStreeam to PeerConnection
... record on server
... loop back to client
... video can be read to Canvas
... no technical measure can distinguish recording from a webrtc call to an arbitrary location
... no matter how many dialogs we pop up
... there's a way to build something that avoids the dialogs

[ Proposed Resolution ]

ekr: what you want to do is often to have the recording on the remote side
... I'd propose not having extra permissions
... and say it's the application's job to deal w/ regulatory requirements
... Chrome and Firefox already render indicators that the media is in use
... we could add a UI for a tape drive
... but i think that should be done in content

Josh_Soref: +1

martin__: +1
... concern that we document this decision in Security and Privacy

PeterThatcher: when you're telling the user that media will be sent
... would it be good for the browser to have fine detail that the media could be recorded

<dom> ACTION: ekr to draft text on no-permission-required for recording for inclusion in the mediastream-recording doc [recorded in http://www.w3.org/2013/02/06-mediacap-minutes.html#action01]

<trackbot> Error finding 'ekr'. You can review and register nicknames at <http://www.w3.org/2011/04/webrtc/mediacap/track/users>.

<dom> ACTION: eric to draft text on no-permission-required for recording for inclusion in the mediastream-recording doc [recorded in http://www.w3.org/2013/02/06-mediacap-minutes.html#action02]

<trackbot> Created ACTION-16 - Draft text on no-permission-required for recording for inclusion in the mediastream-recording doc [on Eric Rescorla - due 2013-02-13].

PeterThatcher: as part of the Chrome

<gmandyam> Clarification: Permission check only on gUM, not for recording is current proposal. This is independent of ekr's proposal about a peerIdentity argument for gUM.

ekr: "maybe this is being recorded and sent to your mom"

PeterThatcher: if there isn't identity stuff
... "maybe it could go anywhere"

cullen: when you turn on your computer, you may be recorded
... and when the green light goes on, you *are* being recorded
... i'm fine with "documenting that recording happens"

Settings and Constraints

<martin__> and the only reason that I made the suggestion is so that we don't have to have these sorts of discussions ever again

[ Dynamic usable constraints ]

burn: for people who want to use media locally
... the constraint mechanism didn't do well for what you had to do in the browser
... Travis spent a lot of time thinking about how this would work
... the original constraint proposal was for selecting a stream
... but there was no ability to change constraints over time

<dom> Constraints and settings

burn: what if you want to change the settings of the camera
... Travis came up with the proposal
... he did most of the work, and gets credit for what's good
... i get credit for what's confusing

[ Constraints mini-review ]

burn: a brief overview
... when requesting media, you can give a set of constraints
... mandatory and optional
... both sections are optional
... mandatory section says
... "if you can't give me a track that satisfies these constraints, then i want an error"
... the optional section is ordered
... these are constraints i'd like to be satisfied, but if you can't, that's ok
... maybe you ask for Width, and aspect ratio
... but maybe Width is more important

ekr: we still have audio:true/video:true?

burn: yes
... i'm not even showing this is for video or for audio
... you have separate constraints for audio and video
... there were differing versions of for audio or video

cullen: can i have a width-min in one row
... and below that a width-max below

burn: yes
... i pulled this from an example
... you can have constraints multiple times (e.g. width-min)
... earlier ones are higher priority

[ Based on Travis's Settings v6 proposal ]

burn: there was a little email traffic, not a lot
... when Travis and I discussed it last week, a few things required tweaking
... some things are slightly different from that proposal

[ The Problem ]

burn: it's called the Settings proposal
... even though v6 removed "Settings" from the proposal
... setting width to a number
... say 650
... is by setting the value, or using min+max to the same value
... here is why we have this proposal
... when you go to get video from the camera
... you could display it in multiple <video> elements
... send it over PeerConnection
... multiple sinks
... multiple Tracks with the same underlying source
... you could have different requirements for the underlying source
... what you want to send over PeerConnection could be a different framerate than what you show
... when you send to a Sink, that could have different characteristics than
... your source
... and the browser has to come up with something to do
... capabilities of source may change dynamically
... something the browser does
... or the user does
... the camera could go away
... or be physically muting it
... you could sometimes want a thumbnail
... or sometimes want a full video
... Sink could change on the fly

[ Home Client ]

burn: Video Camera generating stream of 1920x1200
... two tracks with the same source
... neither constrained
... each track is attached to its own <video>
... one <video width=1920 height=1200>, one <video width=320 height=200>
... the browser will upscale/downscale

ekr: in most examples today
... people don't generally create Tracks
... is the idea we do gUM() once?

burn: how you get different video sources
... vs. the same
... there's some discussion
... the next slide may ... it may not matter that it's two tracks or one

[ Next slide ]

burn: say the 1920x1200 video is changed
... to <video width=1024 height=768>
... the browser will likely scale, as it was for the other sink
... this change doesn't require the source to change

jim: ekr's proposal
... if you set the stream element as isolated
... would the js be able to do anything with it?

burn: you can have a source that's remote, or from file

martin__: tracks that you get back can be manipulated in any number of ways
... by setting constraints
... you just can't manipulate the pixel/audio data

burn: this proposal is independent of that permission discussion

gmandyam: the examples here are downscaling
... if you do upscaling
... and there was a mandatory constraint
... can the UA do postprocessing to enhance the video

burn: upscaling and constraints-mandatory

gmandyam: you have a source
... and the browser wants to avoid making it grainy
... so it postprocesses which increases delay

burn: in this example, there are no constraints set on the track
... that's important for this example
... i'm happy w/ whatever i get from the camera
... and whatever it needs to do
... if it needs to do upscale/processing
... i'm fine with that in this example
... regardless of what the source provides
... and track constraints
... the browser will try to do the most intelligent thing it can
... if the track goes away
... that's an error
... otherwise, the browser will do something reasonable
... if the application doesn't like what it did
... then it can set constraints
... to control things available to the sink

cullen: we had a long discussion a year or two ago
... <video> should be the only thing to do upscaling
... Track won't
... Track could Downscale
... if you duplicate for something that will be smaller
... the Track could downscale
... and as a general rule of thumb
... things should choose largest possible within constraints

burn: cullen is describing guidelines
... not to go into the spec as normative text

cullen: understanding that this is what will happen
... makes it possible to understand that this proposal would work

burn: understand that browsers want to do these things
... this is an interface
... for the application to talk to the browser
... but still allow operation

[ Next slide ]

burn: although the decrease in the largest <video> tag didn't mandate a change on the video source
... it may result in a change by the browser
... if the browser wants to inform the source
... that it'd be better to get a 1024x768
... that has an effect on Tracks
... but not Constraints on Tracks

martin__: there's an Aspect Ratio change
... in the original state
... the 320x200 probably had letterbox
... as a result of changing the display in the big <video>
... you potentially changed the source
... which removes the letterbox in the small <video>
... it isn't intuitive, but it makes a lot of sense

burn: but if you don't like that behavior, you could create constraints for it

[ The Proposal Summary ]

burn: separate notions of "source", "track", "sink"
... Media Capture Streams document doesn't give direct control over source or sink
... only tracks
... source configuration can change at any time
... browser will adjust what's sent to Track
... or have an event

[ The Proposal (Expanded) ]

burn: what's a source?
... Camera, microphone, RTCPeerConnection, file, Screen Capture
... no direct acces or control by app
... browser configures
... what's a Track?
... it's a constraint carrier
... and it holds statistics
... logically, it represents an incarnation of the media
... you could create 2 tracks with different constraints
... if you attach them to a PeerConnection
... there could be different resolutions of media sent on the two
... What are sinks?
... <video>, PeerConnection, Recording
... No direct access or control via Media Capture/WebRTC APIs

cullen: can Track connect to Track?

burn: no
... i'd have to think about what it means

martin__: not on slide
... is MediaProcessing API
... that takes Source Sink and produces a Track ?

hta: in the original proposal
... we could connect Tracks to Tracks
... after scratching our heads for months
... we removed it

Dan_Druta: we haven't discussed <source> being a <video> element
... i think recording should allow saving a streamed item

burn: this isn't an exhaustive list

Dan_Druta: i like the concept of source, track, sink

jim: +1

burn: room full of programmers, no explicit "..." then it isn't there

[ laughter ]

ekr: i have a camera w/ 4x3
... but i have a mandatory constraint for 16:9

<fjh> +1 to concept of source, track, sink

ekr: is it acceptable to pillow box it?

burn: Tracks have constraints
... if you request a 16:9 mandatory constraint
... you'll get a failure right away
... that media can't give you that

martin__: if you want pillowbox
... you'd create it w/o that constraint

burn: it's up to the browser
... if you say you want to receive a video that's 16:9
... a browser may try to be intelligent and do that mapping for you

ekr: if we start moving constraints away
... from sources
... i start to wonder how much i'm being allowed
... do i lose the ability to

burn: having a constraint for `native resolution` might make sense?

custin: i have a 4:3 in
... and i want to munge to 16:9
... by cropping top+bottom
... i ask camera to give me 16:9, i'd want it to do that for me

burn: we'll have to define that in the aspect-ratio constraint

hta: we're descending into details
... i'd like to get to the rest of the slides

justin: looking forward to future discussion

burn: i believe it can accommodate
... there's a lot in the browser that's impl depenedent

jesup: i think it's reasonable when you apply a constraint to a Track
... the constraint could be satisfied by the source
... how much could be impl dependent

cullen: on constraints that imply change to the media
... the standard should be such that browsers are consistent
... if setting a constraint causes media to get twisted around a Torus

<martin__> the thought occurs that mandatory and optional aren't the only categories in this "constraints" taxonomy: the mandatory category might be split into "fail if you can't do this" and "make it this way, no matter what it takes"

cullen: it has to happen the same way in all browsers

ekr: it isn't that i care about aspect ratio

<JonLennox> martin__ - what's the difference?

ekr: i'm concerned that if one way to satisfy is DSP processing
... then how constraints actually operate
... for device selection
... you could satisfy and constraint-A is more important than constraint-B
... but you choose to do DSP on device-1

burn: personal opinion is browser shouldn't do munging from Source to Track
... Track has certain
... it's the Sink that does the mapping

Paul: a little fuzzy
... the browser can control the source
... to make it better suited to the constraints of the track
... what about pan/zoom?

burn: we talked about pan/zoom yesterday
... and i realized it works differently

martin__: pan/zoom might be different
... like the constraints
... must not request processing of source
... if we make it clear that "must not do processing"

burn: i agree
... that's been the mental model i've had the whole time

derf: +1 to martin__
... any constraint that requires DSP'ing the source
... we shouldn't do
... on some platforms it'll be good, and on some, it'll fail

burn: hearing +1s
... anyone who does not want Sources to give Tracks directly what is asked for in constraints w/o DSPing?

justin: can we avoid the double negative?
... is cropping DSP?

cullen: what about Echo-Cancelation?

burn: if source provides something a certain way
... how the source provides it is acceptable
... it's the browser doing it that's a problem

justin: if camera does VGA and i want 16:9, there needs to be a way to get it

martin__: key is browser not doing stuff
... if you want 16:9 from 4:3

<Travis> Let's not forget that there are track constraints and sink constraints (video tag)

martin__: you can push it in a 16:9 <video>

cullen: if i gUM() and i directly connect to <video> it shouldn't need me to pull out and reprocess

hta: we're discussing unobservable control surfaces
... there's no way to discover the difference between to track / from track

ekr: it is

burn: it is with PeerConnection

ekr: what to do w/ 4:3 input and 16:9 display

cullen: i want to send something specific and your camera is different
... i need it to just work
... and cropping or letterboxing needs to be fairly consistent

hta: get through the presentation?

ekr: example case of general principle

jesup: interested to hear what ekr says
... consider native resolution of camera
... may be willing to give any res
... but many will DSP to give any res

ekr: constraints is to prioritizing which device to select

justin: sourceid select a specific device

ekr: i have 2 cameras
... one suitable for aspect ratio i want
... one much less suitable
... first constraint is aspect ratio
... browser decides to rescale video
... choice alg is selected by ordered choice

<dom> Josh_Soref: I'm used to getUserMedia with a pop up allowing the user to select a device that matches the constraints set by the application

<dom> burn: when multiple sources can satisfy all the mandatory constraints, it goes through the optional ones to further filter

<dom> ... if after that there are still more than one, it's up to the browser to select

<dom> ... where the user can be involved in the user

<dom> scribenick: dom

burn: let's finish the slides to look at the peerconnection case

<Travis> There's a dichotomy of desired full-and-absolute control by the app, and variability wiggle-room by the UA. As long as UA can have wiggle-room, then there will be cases where two implementations will be different.

juberti: real world cases of multiple devices is front camera has lower resolution than back camera
... but in many cases, front camera is what you will want
... Are there many cases where resolution matters more than what the camera is looking at?

burn: there are many cases where your app has specific resolution requirements
... if your app cares more about the orientation of the device, then that's what your constraints should impose
... the app has to pick its constraints in accordance with its needs
... there may different behavior for selecting via constraints, and controlling via settings

<Travis> I'm concerned about splitting those scenarios (selection vs. settings changes)

burn: settings is "make it so no matter what", whereas constraints is "only give it to me if it is so"

randell: I'm concerned we're doing a lot of procedures, algorithms, failure cases, for relatively small slice of applications that require very specific device characteristics
... for apps that don't want to let the user pick what the best camera/mike would be

burn: I would argue that applications care a lot about that
... e.g. to match what native apps can do

<timeless> scribenick: timeless

dom: i'd prefer we don't say many apps that do this
... but specific examples
... computer vision is a small UC

burn: how many people have developing Web Apps as their primary job for them or their companies
... I see 2 hands
... be very careful about optimizing for impl
... when we don't have primary users
... my company is more interested in enabling users to build specific apps
... i don't do that
... but the people i work w/ do

adambe: constraints when you select a device v. setting on Track
... discussion about are constraints remembered
... can i read out constraints from the track?

burn: i'll get to that

adambe: when someone sends me a track over PeerConnection
... the PeerConnection is from my PoV is the source

<Travis> We need to finish the slides...

adambe: it's configured some way
... can i read out those constraints from my Track?

burn: let me finish the slides

[ Home Client; Away Client ]

Local source 800x600 video camera

Three local sinks: <video w:1920 h:1200> <video w:320 h:200>

Remote sinks: <video w:150 h:100> <video w:1024 h:768>

burn: there may be constraints set here

martin__: what makes the 1024x768 in the peer connection
... is that the packets from the video over the wire

hta: i'd like to let burn to present 3 more slides

burn: i didn't say if the tracks were constrained or not

[ Next slide ]

burn: video source changesresolution
... 1920x1200
... track content may change, sinks still the same

[ The Proposal (Expanded)]

burn: constraints on tracks, not on sources
... browser does its best to adjust sources to satisfy constraints
... roughly intersecting constraints
... it's the browser's job to satisfy combined constraints set
... constraints can be modified on the Track
... replacing prior constraints
... Tracks can become overconstrained
... 1. when you initially request
... 2. later in process, video source isn't available, or it changes
... in that case, there will be an OverConstrained Event

[ Next Slide ]

[ Home Client ]

video camera w:1024 h:768 fillLightMode: on [at creation ]

<video w:800 h:600> N

<video w:1024 h:768> P

burn: the other track sets a mandatory fillLightMode:off
... it will get an overconstrained event (to the track)
... and the track is muted

ekr: how do i recover

burn: you the overconstrained track set a handler
... and you set different constraints

ekr: do i know which is the problematic constraint?

burn: the intent with the current spec is to give you information

cullen: if N and P set them both to On
... and then N sets it to off, it will fail?

burn: yes

cullen: last guy who tries to make an inconsistent change to the resource loses

martin__: that doesn't preclude locking discussion from yesterday

burn: that's why i asked about between tabs

martin__: this would work ok w/in an app
... and ok between tabs in non-exclusive

burn: "no required side-effect on source"
... P has a res of 1024x768
... N has res of 800x600
... P was getting nothing
... but camera was still generating 1024x768

[ Next slide ]

burn: there's no requirement that the browser do anything
... but the browser may change camera setting to match remaining sink
... up to browser policy
... permitted but not required

martin__: you were concentrating on optimization
... i thought it would end

burn: muted

Josh_Soref: it matters because it can resume

burn: v6 proposal said overconstrained
... caused all tracks to go into an ARM (?) state
... here, the right thing is that the track causing overconstrained to muted

martin__: softer, you try to set
... and instead it fails w/o muting

<Travis> Constraint application cannot fail.

<Travis> It is an overlay set of requirements

burn: i like that

adambe: i like martin__ 's proposal
... that when you set something that doesn't work

<ekr> I also like martin's proposal

adambe: it discards the latest setting and continues

<ekr> also martin_______'s proposal

adambe: when something happens and it isn't something that you did
... when the source changes
... in some cases, it might be nice to pop a constraint

burn: say the camera is unplugged
... all your tracks are severely constrained

stefanh: aren't they ended?

martin__: ignoring shit happens

burn: if there's a switch on the camera that's on/off

Josh_Soref: it might be a "mute"

stefanh: i think we have an onmute event

martin__: you may not have software control over mutebutton

burn: if something happens in the world
... we need to have a way for things to go to muted
... instead of just going away totally

martin__: i don't see any reason not to allow to continue to receive the stream

burn: user will see something

martin__: you listen for event,
... and if you care, fiddle

burn: don't change track

[ Coffee Break 15 minutes ]

[ The Proposal (Expanded) ]

burn: we have Capabilities, and the current state
... capabilities and states are properties of a source
... - not a track

ekr: same namespace for keys?

burn: yes
... on thing that might change is that Selection v. Control
... for Control case, you might want current state to be on Track and not Source
... your Track might be requiring DSP munging
... we use the term "released"
... "once a source has been `released` to the application (via permissions, pre-configured allowed, or other) the application will be able to discover additional source-specific capabilities."

[ The Other Details ]

burn: the constraint registry talks about min-max and enums
... this proposal also includes primary types (strings, ints, float, boolean)
... i'm tempted w/ adjusting the registry
... there are also source ids
... i got a source id the last time i was here and saved it
... you may also be able to get "video", "audio" and the complete set of source ids
... proposal has separate types for audio and video
... and explicit creation of Tracks (not currently possible in gUM)
... this proposal adds an explicit constructor for Audio/Video tracks
... which you then call into gUM

[ Proposed (mod of Settings v6) ]

burn: it talks about tracks being audio, video, "none"
... that a track can be "readonly" or "remote"
... you can have "readonly" "video", or "remote" "video"
... readonly/remote are other states
... we're not sure exactly how it would be used

martin__: not a particularly good answer
... i think you need to go into more about readonly
... if you insert it, you have to justify it

burn: Travis inserted it

martin__: i can't write video to my camera

burn: readonly is intended to mean you can't change settings on a source

Josh_Soref: one case is that it's a file stream
... but that discloses it's a file stream

martin__: i'm not sure we need it
... if we don't kill the track for applying settings
... then i don't know we need it

burn: ok

[ About the proposal ]

burn: there were a small number of comments to the ML
... it's a long very detailed proposal
... probably very few people read it
... i wanted to talk about it
... stefanh said that the editors would start applying
... but cullen shot it down

martin__: since cullen isn't here, i think it's very good

stefanh: cullen made that comment after reading the wrong version

martin__: there's a huge body of stuff that's very useful
... we can fix up the little bits later

burn: i agree, although we need to work out selection v. control
... it's the only thing major enough to work out

ekr: 1. is this the right API
... 2. how do we feel about the specific field values
... suggestion is we take these in two stages
... 1. do we think it's the right arch?
... 2. do the list in pieces

burn: i agree w/ you
... my plan for today
... was see if we can agree on Arch
... then start applying that
... if we agreed today
... we'd go on to the capabilities list
... not try to agree on things to go in, not even describe all of them
... these are many that have been proposed
... i wanted to poll for levels of interest on these
... to help me order for a start of a discussion

[ About the proposal ]

burn: open the floor to comments

PeterThatcher: lot of complexity
... concern that browsers do the same thing

<Travis> It will be very very hard to get exact browser parity when there is wiggle-room in terms of how to handle unconstrained tracks.

PeterThatcher: hard to see the browsers doing both right

burn: do the same

cullen: words i have are "have the same UX"

PeterThatcher: has there been consideration for a lower level api?
... less magic in the browser

burn: let me speak about history
... discussed for a while
... 2 camps of people
... those who believed the browser should make decissions
... those who believed JS developer should make decissions
... constraints was an attempt to find a middle ground
... let JS developer specify at the level that matters
... and let browsers be wise beyond that
... it seems there's a significant camp who want more control after selection
... that led to the proposal of a settings api
... it's still a settings api for control after selection
... but it still uses constraints language
... so it still gives flexibility
... one other aspect here
... privacy/fingerprinting
... there was concern initially that giving the JS dev direct complete access
... ideally from a App dev
... they want complete list of devices and take absolute control
... we heard from a number of browser devs saying that isn't a good idea
... if we relaxed our notion of what we can give to devs
... or if we came up w/ a permissions set
... to give more control
... i think this is the best compromise of what you give the App dev
... to allow control
... and to let them to be notified when it can't be
... App says "if you don't give me this, i can't run"
... it's a long winded answer

jim: Recording also has Capabilities and settings
... Recording doesn't have Constraints

burn: what if you have multiple recorders?
... you could say you have constraints but none

jesup: you talked about how we got here
... largely fingerprinting and control

burn: and selection

jesup: before user gives consent
... concerned about giving full details of devices
... propose an alternative solution to that conundrum
... allow JS to have access to device list and capabilities
... within a JS worker with a different origin
... it's constrained to return an output of a sourceid

burn: i'll let ekr comment

jesup: you rate limit how many times you call it
... to avoid bit yielding
... you remove a very complex hard to make sure it's going to work the same way
... i'm concerned about implementing this and testing it
... and making sure it works in configurations
... i'm tempted to give app writers
... and say to app writers "we have this interface, don't use it"

ekr: jesup's proposal could work
... leak of 20-30 bits
... limit to 2 queries per second
... we're done in 8 seconds
... not sure it solves the privacy problem, not sure i care either

cullen: i think we're coming up w/ a lot of stuff that works for settings
... dealing w/ tracks and control them once you have access
... the selection part is where we're stumbling more

burn: it's been in the document longer

cullen: you can see the widespread agreement </sarcasm>
... generally, it's about the user choosing the camera they want
... i wonder whether we should ...
... the privacy history
... i'm becoming
... no one has ever identified attacks
... once you could do Fonts on Canvas
... as we go down this path, i'm trying to understand why we're going down that hole

burn: i hear you
... adding settings is more complex than selection
... i don't think we have new arguments against selection different from when we started
... it definitely should make the easy case easy
... that's still easy
... it handles intermediate case nicely
... you don't need device ids
... or a stored database of every camera on the planet
... i'm nervous about chucking it and starting over

martin__: jesup's suggestion
... other than ekr's bits
... means you can't build the UX we're talking about
... w/ a source picker in content
... with the rate limiting of the worker
... on cullen's point, i want to think about it

burn: the people arguing for settings control aren't here

cullen: the model here for settings seems ok
... but selection seems less understandable

ekr: why i like having selection
... 2 working examples in browsers
... Chrome... there's a browser global camera preference
... do you triage?

justin: we pick the global camera pref

ekr: firefox prompts you all the time
... 3 models for how selection should work
... #1 it's a browser setting
... #2 it's a user controlled setting
... #3 it's a JS-Browser negotiation
... #4 in content selection
... this quasi magic thing isn't what people want
... Control in Chrome, in Content, in neither?
... in content selectors have these privacy risks

Josh_Soref: it seems like everyone's happy w/ merging Settings in
... but we might want to drop out the Selection bits

hta: we have action items about Selection from yesterday
... but today we're talking about Changing Constraints aka "Settings"
... once the line empties
... (if ever)
... i'd like to call consensus to incorporate this version of settings manipulation into the document
... knowing that how we do selection is still dependent on Action Items from yesterday

jesup: i'm sympathetic to that we've put a lot of time into this
... a lot of effort
... you don't really control that
... you can't apply that constraint on a mac

<fluffy> OSX computers will give you whatever resolution you request regardless of capabilities of camera

jesup: "i know from a DB this camera can't do that"
... my concern about selection constraint
... is people will overspecify, locking out the user
... even if it isn't the perfect camera
... i'd like to really understand UCs that drive those
... to override user's ability to choose

burn: bad apps can always be written
... bad apps won't be used for very long

Dan_Druta: i'd like more clarity
... no guarantee about control

burn: all tracks connected to source receive an overconstrained event
... we had a discussion about different option
... i liked proposal of not killing-muting track and just sending event

Dan_Druta: for some it works, for some not
... but js app doesn't have access to this

<Travis> In the v6 proposal, there is actual no "state change" event. So, if no constraints are applied, there would be no notification of source state change (short of polling the state values).

<Travis> (This is probably a missing piece in the v6 proposal).

Josh_Soref: the path is disappearing from file content widget
... partially UX and partially security

PeterThatcher: we have consensus on Settings
... does that include cropping?

hta: we've gotten enough info that cropping -- how to adjust video to fit into boxes
... requires more discussion

<dom> dom: for in-content device selection, couldn't browsers provide a tainted in-content widget that JavaScript couldn't access? similar to the file selector widget

ekr: stop improving Chrome

hta: that's life
... Queue is empty

<ekr> it's too awesome already

hta: show of hands

ekr: voting by individual or company
... if i were recording, it would require me to record by company
... if knowing we have an ongoing discussion about:
... A. Selection
... B. Cropping
... C. we haven't discussed specific constraints
... do we have consensus to incorporate the framework of Constraint-Manipulation-After-Selection into the specification?

<Travis> Travis: Yes, we should incorporate!

ekr: if you believe we should incorporate this, at this time
... raise your hand
... consequences of yes/no?

hta: if you say no, we have to discuss this more later
... and you have to propose changes
... and if not, your proposal for "deleted"

ekr: if consensus is yes, a new ED will have this incorporated

gmandyam: is this v6 compatible?

hta: it will be self-consistent, and similar to v6
... there were a number of minor tweaks between v6 and now
... v6 was impossible to implement

cullen: burn presented a number of substantial differences to v6

burn: correct
... my goal as an editor in putting this in
... for areas where we don't have consensus
... i'll include a big note that we're still discussing the section
... if i miss something like that, please call me out on it

ekr: just wanted to understand that

dom: question about integrating it?
... or is it about consensus?

Josh_Soref: which work-mode do we have?
... propose, reach consenus, commit
... or put in and then get consensus

burn: minor things may not have a note
... major things will have a note until consensus

justin: for things that need more discussion

hta: let's do that after this
... if we try to do too many things at once, we won't do any well
... If we think the editing team should incorporate the proposal
... knowing open items

[ 16 hands + Travis ]

hta: if you oppose, raise your hand now

[ 1 hand = justin ]

hta: editing team will incorporate as stated
... chairs take an action item to ensure open items are discussed on list

cullen: justin, what are your concerns?

martin__: i'd like to get justin's opinion
... i'd like editors to mark open issues in the draft

burn: that's the intent

hta: yes

justin: all things people want to do
... are all things because they're trying to do these things
... we have all these things
... but we might have many more as we try to do the rest

hta: next step is incorporating

burn: if it weren't for privacy, we'd be done a long time ago

ekr: this spec is far in advance of anything impls have done
... we're going to be finding things we don't understand for quite some time
... because it's so far ahead of what we've done

justin: getting stuff working is focus of work
... that list of constraints
... i'd like to make a white list of things we're going to care about

burn: that's the next slide

[ Candidate constraints/states/capabilities ]

burn: as an editor, we haven't had specific ones to talk about
... we talk about specifics, it becomes general, it goes back and forth
... i'd like to with the group of people in the room
... not the same as list
... what i'm going to ask
... go through each one, and with a rough idea in your head of what it means
... is it something that might be good to have as a constraint
... to see if there's some breakingpoint
... a big groundswell, v. only a few
... it's input to the editor team for a small candidate initial set

justin: for `now` as opposed to `ever`

burn: what can be proposed to go into a first draft

hta: we're delaying WebRTC until tomorrow morning

cullen: take AspectRatio to near Zoom
... do sourceid, width, height, framerate, facingmode

burn: i don't want each person to give me their list

gmandyam: that list is my fault
... it wasn't meant to be for everywhere
... i was satisfied w/ the v6 list
... i hate reopening this debate

hta: negotiation isn't between you and Travis isn't consensus of TF

burn: Travis's reason for excluding AspectRatio wasn't valid

dom: is this about settings or constraints?

burn: yes, constraints=states=capabilities
... we're only talking about control

cullen: how are we doing this?

Josh_Soref: hands up, one at a time

burn: +1

cullen: object to conclusion

Josh_Soref: input to the editors

burn: a proposal may go to the list, but no change

[ sourceId 13 ]

[ width 7 ]

<dom> [so people understand what sourceId means in the context of control?]

[ height 17 ]

<dom> [but then that's selection, not control]

[ aspectRatio 10 ]

[ framerate 17 ]

[ facingMode 12 ]

[ zoom 1 ]

<Travis> dom, sourceID can be applied as a constraint to a track with a different sourceId. That's an overconstrained situation :-)

burn: would anyone actually raise their hand for anything after zoom

[ Candidate constraints/states/capabilities ]

<dom> Travis, but then that's not one useful to include in the list of control constraints

[ sourceId 18 ]

[ volume 16 ]

[ gain 1 ]

burn: thank you

dom: Selection is ?

<Travis> dom, The goal being we maintain parity between state, constraints, and capabilities.

hta: action item on chairs to take to list

[ Lunch ]

<Travis> Folks should spend the time to read the v6 proposal in detail if they haven't had a chance. Thanks!