See also: IRC log
<trackbot> Date: 14 November 2013
<dom> Scribe: gmandyam
<stefanh> minutes to approve: http://lists.w3.org/Archives/Public/public-media-capture/2013Sep/0232.html
Stefan: Minutes approved.
<dom> Agenda
Stefan: Agenda bashing ...
... Will cover Image Capture and Recording towards end of
meeting if time permitting
<silvia> http://www.w3.org/wiki/images/a/a6/LC_ambition.pdf
<fluffy> http://www.w3.org/wiki/images/a/a6/LC_ambition.pdf
Stefan: Dec. 6 is planned for issuing CFC on Last Call. Last Call period would be from Dec. 15 to Jan. 15.
Dan: How does the change in process discussed at Plenary apply?
Dom: The new process will not be applicable to this document. It won't be ready in time.
<dom> Proposed revisions to the Recommendation Track process
Dan: FYI for remote attendees: at the Plenary yesterday a new process was discussed where Last Call is redefined so that it comes after final review.
<silvia> (presentation given about changes to REC track process)
Dan: I have not walked through the document to see what all needs to happen to proceed to LC. The intro of constrainable I/F has been the biggest change, but a lot of the remaining text is not aligned with constrainable. This will confuse reviewers.
<dom> Open Bugs on Media Capture and Streams
Dan: How to obtain source
settings is an item that needs to be taken up. The satisfied
constraint may not be consistent with the source setting.
... Sent out a native states proposal to the list. Seems to be
consensus within the TF.
Adam: Have been working on
resolving bugs. There was a call to create bugs. Have been
going through that list and resolving.
... Editors have ordered the bugs into two categories: (1)
Ready for inclusion n the spec, and (2) needs more debate.
jib_: Have concerns about constrainable.
Dan: Will go over constrainable, including whether Peer Identity/noaccess are in fact constraints.
<dom> bug on "noaccess / peer identity as constraints"
Stefan: Is the schedule realistic from the editors' perspective?
Cullen: My suspicion is that LC will result in a large number of comments, many of which have been put forward to the list before. But I think we should still push forward for LC.
<fluffy> http://www.w3.org/wiki/images/f/fa/BurnettConstraintsTPAC2013.pdf
Stefan: Moving on to Constraints and Constrainable ...
Dan: This is not intended to be a discussion as to whether constraints are useful.
<jesup> I agree with Cullen - we will get a lot of "oh, now that I read it" comments and "I made a comment 10 months ago and no one replied"
Dan: It may not be appropriate to
use constraints in all places (e.g. in WebRTC). This is
confusing to newcomers to the specifications.
... We will have a review in WebRTC. We will have to do the
same thing for Peer Identity and noaccess.
<dom> Dan's slides on Constraints
<fluffy> On slide 1
Dan: (Slide 2) Constrainable properties represent a cross between what the UA needs/offers and what developers require.
<fluffy_> On slide 2
Dan: (Slide 3) Web success is due to "browser default intelligence".
<fluffy> slide 3
Travis: Browser is flexible enough to take any input and try to make sense of it.
Dan: (Slide 3) Developers using
gUM need to work with shared devices. For WebRTC, networks may
be unpredictable resulting in negotiation failure or transport
failure.
... (Slide 3) With flexibility implementers can offer improved
performance both in capture and networking.
jesup: adjusting resolution arbitrarily has impacts particularly to mobile performance.
jesup, please enter in IRC if I've oversimplified or gotten it wrong
Dan: (Slide 4) app developers need predictability/controllability for their app
<jesup> jesup: Adjusting resolution automatically is especially useful if you have a mobile device, since the overhead at capturing at 10x the pixels needed wastes CPU/power and may limit frame rate
Dan: (Slide 5) Each capability in the constraint approach has a range of acceptable values, whether an error event must be raised if the contraint cannot be met ("Mandatory"), and ordering of optional constraints based on priority.
<dom> gmandyam: if the mandatory constraint is set to 419x299, should be the browser throw an error event if it can only do 420x300?
<dom> burn: yes; a developer can use min/max values instead, but if the dev requests a mandatory value with that specificity, he'll get what he's asking
<dom> gmandyam: that confirms my understanding; this will require a lot of developers education
richt: Mandatory constraints are
not best effort. It is all or nothing. There is no
fallback.
... The idea that it is better to fail is an exclusionary
model.
... This is a "first world" type of approach.
<dom> jib: you qualified what developers get from an unmet constraint as failure; but that's not exactly how this works with mandatory constraints
<dom> ... what you get is an error event later on
jib_: Want to clarify that "failing" upon inability to meet mandatory constraints is not really "failing." How do we deal with unknown mandatory constraints?
burn: We should discuss unknown mandatory after we finish the constrainable discusion.
<JimBarnett> No, not quite, if applyConstraints fails, you get the error callback
<JimBarnett> And the old constraints stay in effect
adam: Do unknown mandatory constraints result in an error or an event?
burn: If the mandatory constraint cannot be met upon invocation on gUM, it is an error. At a later time, it would be the overconstrained event.
fluffy: "First world" is inflammatory language. Lots of people raised good use cases for this.
JimBarnett: If there is a failure to apply a constraint, then no changes occur to the existing device state.
<dom> [slide 6: sample Use Cases]
burn: (Slide 6) Sample use cases - broadcasting US Presidential address, door security camera
<dom> (the presidential broadcast seems a stretch as a use case: here the user and the application developer are presumably tightly coupled)
burn: (Slide 7) Use constraints when browser does not have direct or exclusive control over capability. Use direct configuration when browser has direct and exclusive control over the capability. An example of direct config. would be in WebRTC: offerToReceiveAudio/Video.
<richt> fluffy, my point was not to be inflammatory but to highlight that it is easy to overly constrained media capture when you're sitting on a T1 connection with the latest hardware and software at your disposal. i.e. developers are not necessarily the best judges of what their users are capable of. But point taken.
Travis: Is a direct config. the same as defining an API method to invoke?
burn: A direct config. is a normal setting.
Dom: Direct config. is the same as non-constrained approach.
burn: Browser in WebRTC use cases may need to take action more quickly than would be allowed with API control (e.g. congestion control).
Dom: Peer Identity and noaccess are not constraints, based on this definition.
jesup: It would be useful for people to come up with media capture examples as opposed to WebRTC examples. For example, noise suppression is something you just turn on and don't think about.
burn: Noise suppression would probably be a setting if it is simply an on/off feature. But different levels of noise suppression could lend itself to constraints.
adambe: Constraints should include capabilities, range constraints, and settings (direct config).
burn: I think this is heading
into the mandatory unknown constraints discussion, which we
will take up later.
... (Slide 8) types of constraints include property value
range, and enumerated list (e.g. of source ID's or
booleans)
brun - > burn
burn: (Slide 9) Describes difference between capabilities, constraints, settings
Travis: Going back to door security camera example. Given that we have capabilities, we have no ability to express a preference without constraining.
burn: The way to specify preferences is by an ordering of optional constraints.
Travis: We should put an example in the spec. It is a good feature.
DanD: Constraints specify a distinction between "needs" and "wants", and "buy" i.e. you get what you pay for.
silvia: I assume the idea of this presentation was to explain constrainable. I would like to see code examples.
burn: I didn't want to go over
syntax in this presentation.
... This mechanism was present before the constrainable
interface was spec'ed.
<dom> gmandyam: imagine I have a physical device that the end-user can directly control
<dom> ... the app developer uses the constraints mechanism to adjust a setting the user can control physically
<dom> ... once the dev gets the stream, the user fiddle with that setting
<dom> ... the app dev will then get an overconstrained event
<dom> Travis: this means you have to threaten the user? :)
<dom> Burn: you can't stop the user from doing that
<dom> ... no matter what mechanism you use
<dom> dom: mandatory constraints requires tight coupling with the end-user
<dom> ... as the US presidential broadcast (stretchedly) illustrates
<dom> ... we should make it clear to developers
<JimBarnett> there's no reason that you can't specify an enumerated set of values for a range
jseup: One of type of request seems hard to specify. You need a set of values that are a subset of a range and are known (e.g. QCIF, or codecs w/fixed resolution sizes). It is going to be hard to specify. fps would not be suitable for a range, but more to a set of acceptable values. Optional does not really allow this.
burn: We could move beyond min/max and enumerable types of contraints to handle this.
anssik: We have one real-world use case: exposing depth stream over gUM (e.g. for multiple cameras). I would like to see how we can facilitate extensions in the future. We want the base functionality to be shipped ASAP.
adambe: Are you going to discuss
the list of constraints that are mandatory to be
supported.
... The mandatory vs. optional is really the tough debate. We
should just focus on the list of constraints. There is
consensus on use of constraints.
fluffy: We should sort out future mandatory constraints. I think the implementation should fail any unknown mandatory constraints. Our advice to developers would be not to use mandatory constraints if you don't know what you are doing.
jib_: Dictionaries are by definition designed to ignore unknown keys.
cullen: WebIDL doesn't bother me. Let's say in the future we extend to 3D cameras. Assume you request as mandatory a 3D camera when the browser doesn't support the constraint. You want it to fail.
jib_: There is another way. The browser provides a method that gives information on what it actually supports.
\jib_: You should not use mandatory as a developer unless you have a fall back.
jib_: This not good for novice developers.
<richt> well, seems that I completely agree with jib_
dom: Most of the tension is on
mandatory vs. optional. Future mandatory is not a separable
issue.
... Maybe the natural way is to make it harder to request a
mandatory constraints.
<fluffy> +1 on default thing is an optional constraint and need to do something extra to be mandatory
<richt> I can see that working dom.
burn: I don't mind making it more "difficult" to request a mandatory constraints.
<richt> ...pending a proposal
<JimBarnett> I'll try to think of the appropriate syntax to make mandatory constraints harder
<jesup> +1 on optional being the default, mandatory needs extra work/parameters/etc
<Zakim> dom, you wanted to say one way would make it to make mandatory constraints *harder* to request
Travis: Mandatory and optional should not be at the same level of "easiness".
adambe: I like this direction. We should pursue a default of optional, and something extra must be done to achieve mandatory.
Chairs: We need to see proposals.
<JimBarnett> One thing to do would be to require a constructor for mandatory constraints
richt: This is a good direction. Default use of mandatory could mean that there is a good chance your app is unusable, which is not the way of the web.
silvia: I like the mandatory part. I don't think optional is sufficient. The browser will be conservative when specifying a range of values.
JimBarnett: 3rd paragraphs of Ch. 10 provides an example for Silvia.
silvia: A special value denoted as "best" would be better than an ordered list.
<jesup> I don't think width:1920, width:1280, width:1024, ..... is a great way to say "max" :-)
jib_: Mandatory constraints should be WebIDL dictionaries. The current spec is not consistent with the WebIDL specification.
<jesup> I should note jib has been wrestling on how to implement constraints within the browser and make it interface to the DOM and WedIDL code
<Travis> Silvia brings up a good point--that Chrome currently is not giving "best effort" as the default, but rather 640x480.
<Travis> That's why she want to set "best effort" as a preference.
<burn> travis, silvia, yes, so the interesting question is why
<Travis> I think the spec should make "best effort" the default--so that it will be clear that Chrome has a bug.
jib_: The implementation needs to provide a method as to what mandatory constraints it supports.
<jesup> audio is good now
fluffy: Re: getting the max video resolution. We agreed that absent constraints, the browser would provide max resolution and fps as default. We should provide advice as to what the browser should do with no presented constraints.
Silvia: Implementations like Chrome don't do that today. By default, we may not want post highest res video on the internet.
<jesup> cullen: what if max frame rate is at 640:400 @ 120fps, and max resolution is at 1920:1080 (@5fps)
dom: There is a question as to what to optimize first, absent provided constraints.
<fluffy> what type of phone are you using on shenzen? is it a voip connection?
<jesup> can't optimize both, so the behaviour is poorly specified - which may be ok, but we should be clear
We are going on break in Shenzhen until 11:10 local time
<silvia> also, if there is agreement that lacking constraints the browsers should provide the best, then that needs to be in the spec
<jesup> frame rate preference (up to max X fps) versus quality(resolution) preference (down to min X fps) is common for apps to want. And not it's not just an optional "frame_rate: best"; the camera might do 240fps and you may not want more than 60.
<jesup> configuration/preferences for this sort of stuff has enough edges cases to fill a dozen ratholes ;-)
<dom> ScribeNick: dom
burn: summarizing from before the
break
... there is agreement that the constraint approach is
workable
... one suggested modification (lacking concrete proposal yet)
has been to make it less obvious/less easy to request mandatory
constraints (vs optional ones)
... another suggestion has been to indicate they want some
extreme value of an actual property (e.g. max width without
knowing what that actual value is)
... both of which are doable, hopefully not requiring a
horrendous effort
... still some uncertainty about how to deal with mandatory
constraints for unknown properties
stefanh: that matches what I heard
adambe: in the spec we have a
limited amount of space for constraints
... would it be useful to list examples and how they're met
with use cases
... e.g. an appendix
... this would also help ensure we can achieve these use
cases
burn: sounds good
gmandyam: I've assumed that
constraints defined in the spec are mandatory to be
implemented
... is that everyone's understanding?
burn: yes
dom: we will have test cases to verify that
jesup: what does support mean in this case? is it "recognizing"? does it need to do something with it
burn: do as defined
jesup: ok; that might be hard to test
dom: I share this concern; I think we can do a reasonable job to it, but we need to understand that adding something in the registry for constraints name should come with real implementation experience and testing
burn: the spec currently does 2
things: it defines a set of constrainable properties and
requires implementation of those
... we haven't made a decision about what the conformance
requirement for properties in the registry
... we should discuss registry and impact on conformance to an
upcoming meeting
fluffy: this is clearly something we need to discuss before we can imagine moving to Last Call
stefanh: we'll put it on today's agenda then
fluffy: dealing with unknown constraint properties is also something we need to tackle at the same time
JimBarnett: constraints are
supposed to deal with the fact with varying capabilities
... which makes it hard to pinpoint what "support" means
... e.g. for a property that a given camera supports
burn: based on discussions at the
break, we also need to look at permissions again
... I think we're slowing converging toward an
understanding
... but we're not completely there
... what is available before having permissions (and permission
for what)
... we need to have that review done
silvia: I want to add to the
summary about the "best effort" outcome
... with the possibility for a constraint to give the best
possible value
... I would suggest having special values e.g.
"maxPossible"
jib: +1 to permissions
review
... esp in light of gathering info for or by setting mandatory
constraints
fluffy: I would like to clarify whether a browser without a given hardware can be conformant to gUM
dom: the algorithm already ensures this
<jesup> I
<jesup> I'd like to see getUserMedia available on browsers/machines with *no* cameras or mics :-)
JimBarnett: Constrainable will
have to change a lot in light of our discussions
... so probably not worth doing this now
-> http://www.w3.org/wiki/File:Life_cycle_session.pdf Adambe's slides on lifecycle
adambe: [slide 2: mediastream
state - recent changes]
... ended has been replaced with inactive as we moved
everything to tracks
... mediastream is just a container whose state is a
convenience shortcut to the collective state of the
tracks
... reflecting whether any contained track is active
... should we switched to a positive (active) state instead of
the current "inactive" one?
<scribe> ACTION: adambe to switch MediaStream.inactive to MediaStream.active [recorded in http://www.w3.org/2013/11/14-mediacap-minutes.html#action01]
<trackbot> Created ACTION-25 - Switch mediastream.inactive to mediastream.active [on Adam Bergkvist - due 2013-11-21].
adambe: [slide Stopping
sources]
... in which context does this stopping/granting happens
... when you stop a source, that said source could be used e.g.
in several tabs
... what should happen when you switch it off in a single
tab?
... the scope of stopping a source should be in the specific
context in which it was created
... once you've been given access to a source, you can get as
many tracks as you want
... this is the context in which you can stop a given
source
... stopping a source in a tab doesn't stop it another
Giri: so this isn't linked to origin, but to tabs
dom: this is what is defined in HTML5 as the browsing context
adambe: we should explicitly say so that people don't have to think about this
JimBarnett: what happens when you
have a track and lose permission to the source (or the track
hasn't been attached yet)
... what should getCapabilities() return in that case?
adambe: good point; but we'll
need to look at this separately as part of the permission
discussions
... [slide MediaStreamTrack.stop()]
... revoking access to a source is done via the track the
source is connected to
... if you call the stop() method on the track, it will stop
the source of the track
... and it will end all the connected tracks
... it's been requested to make it friendlier to other
tracks
... my counting the number of references of a given source
across tracks
... if another track is created (e.g. via cloning, or a
constructor, or a new call to gUM)
... every track instance would add to the reference
counter
... and it would be decreased on garbage collection or
collecting stop()
silvia: what's the point of sourceId in that case?
adambe: it's a way to remember and recall a previously selected camera
travis: it's a persistent session
identifier
... if you stop a track and the source that is referenced from
another track
... does the MediaStream gets active=false?
adambe: yes (cf slide 2)
... [slide MediaStreamTrack.stop() #5]
... intention of the stop method is to make it possible to
develop well behaving application
... letting them to revoke no longer needed permission)
... the application itself could be dealing with it on its
own
... but not in all cases (e.g. if you're using an external
library)
... so the current stop() is not nice, ref counting would be
much better
<scribe> ACTION: adambe to write ref-counting mediastreamtrack.stop() [recorded in http://www.w3.org/2013/11/14-mediacap-minutes.html#action02]
<trackbot> Created ACTION-26 - Write ref-counting mediastreamtrack.stop() [on Adam Bergkvist - due 2013-11-21].
jesup: I generally agree with the
functionality if that's indeed we intended
... I don't know if ref-counting really comes into it
... if you simply rewrite it as stop() applies to Track
only
... this will do the same and be less confusing to refer to ref
counting (which only applies to dev visible objects)
adambe: makes sense
<jesup> ack
adambe: [slide 6
"Proposal"]
... if we really need an explicit stop on sources, we could add
a method to MediaDeviceInfo
<jesup> I see no need for it
adambe: we don't have a clear need for this (and MediaDeviceInfo can't take a methods)
gmandyam: what stopping a source implies? turn the camera on/off?
adambe: this is not defined given that we don't know we need it
silvia: agree
<silvia1> lets not have side effects from one tab on other tabs in our browser
<richt> ScribeNick: richt
<richt> <agenda bashing>
fluffy: If the browser comes across a mandatory constraint it doesn't recognize what should happen?
burn: With permissions let's discuss what information is available in the permissions check process.
jib_: I believe strongly mandatory constraints should be WebIDL dictionaries.
jib_: Which means anything not understood by the browser can be ignored.
jib_: Anything the app doesn't know about it can use getCapabilities to discover.
fluffy: Why should it return an error to things it does understand but not to things it doesn't understand
jib_: Conflating two things...
<burn> this is related to the registry -- we can't put all possible registered constraints into WebIDL in the gUM spec
burn, you could argue that the registry should be the WebIDL in the spec
fluffy: We're discussing things that have been agreed for a long time in this working group.
jib_: A lot of people want to be able to say: 'Please give me the face camera'
fluffy: If they want to see all the cameras then they won't put those contraints in.
fluffy: You're taking the position the app cannot control the experience.
adambe: When we said there should be the concept of mandatory and optional but we were discussing before lunch that perhaps it's the apps job to monitor constraints.
adambe: You mark some and force mandatory behavior on those.
burn: Don't want to get in to a full proposal now but...
burn: You have a preferences list, essentially. You put your constraints in priority order but then there's a separate call. The separate call would allow the developer to indicate a mandatory constraint with a callback if the constraint can't be satisfied
adambe: We discussed that when you declare mandatory constriants there too easy to declare.
adambe: And we discussed making it harder to declare mandatory constriants.
fluffy: You have wants and needs. This affects what you expose in the user dialog box.
fluffy: The 'wants' are things that displayed that meet that. The 'needs' you want an error if you can't fulfill them.
dom: Even when there were mandatory constriants there was feedback from implementors that it was difficult.
fluffy: What are our requirements? We need to meet those use cases.
dom: I don't know that the use case says anything how web app developers enforce this.
dom: The question that emerges is: What kind of discussion can we have in the absence of a proposal for mandatory constrinats discussed this morning?
fluffy: I understand that. But what we're discussing is what the basic requirements are.
fluffy: Are we hitting the reset button on this?
jib_: I don't think we are proposing that.
fluffy: Would like clarification from the chairs.
stefanh: When Dan presented this morning people liked it a lot.
stefanh: One complaint was that it was too easy to use mandatory constraints.
stefanh: We can design around that.
jib_: We make it harder for the advanced user but easier for the simple user to get along with this.
jib_: First you have to figure out what the browser supports. Best way to do that is to ask the browser what it has.
jib_: e.g. do you have 3d camera capability?
fluffy: I have a call that let's me do that. I understand that.
fluffy: What you're describing if I ask for the front facing camera then I want it to display the front-facing camera or nothing.
fluffy: You're saying your implementation won't do that.
jib_: That's not what I'm saying at all.
burn: We're talking about unknown constraints so let's discuss that.
fluffy: If I put in mandatory and there's no front facing camera what happens.
fluffy: But you're saying you will present to the user all the cameras in that case.
jib_: What we found was users tend to disagree with what apps want.
jib_: The implementation will go through the array and try to match the constraints.
jib_: It penalizes someone that has more than one camera.
jib_: If an app asks for the back camera it will work on person A's device.
jib_: But won't work on person B's device if they don't have a back camera.
jib_: My opinion is that the user should be able to override that choice.
<jesup> And in this case, the app *said* it was optional, but the result was to effectively make it mandatory because the user can't select anything that doesn't match it
fluffy: Suggest you write that up and send it as a proposal to the working group.
<Travis> Can someone summarize?
jib_: The way to think about mandatory is like a flamethrower.
jib_: For the novice user getting an error is not good.
fluffy: I have no problem making it inconvenient to use mandatory.
fluffy: But we still need to make it possible to use for the advanced user.
jib_: We have a new constraint that's not mandatory to implement then you should check with the browser that it supports it.
burn: One of the things you've said jib_: The apps needs to check what's available re: constraints.
burn: We actually need our permissions discussion. What information is available before you access the device and what is available after?
burn: Let's say you want the front facing camera. If the browser cannot ensure I get that camera I may not want to show it.
burn: maybe gray it out.
burn: So we need to have the permissions discussion because it's related.
dom: That's not something you can determine for e.g. standalone webcams.
dom: What application will not even try to fulfill the use case in the absence of e.g. a front-facing mandatory camera request?
dom: It's not just that mandatory constraints change the UI but also change depending on whether the browser can determine about the camera devices.
burn: In order to answer this question we need to discuss permissions and extensibility via the registry.
jib_: Confused what permissions has to do with mandatory constraints.
dom: I agree with that.
fluffy: I agree with that.
jib_: Should be a convenient enough that when novice users use mandatory constraints there is a reasonable fallback available.
burn: In response to that, I'm not convinced on the 'this is how the web works' argument.
burn: Might be interesting to add ability to determine what constraints a browser might know about.
burn: Sounds like the registry discussion.
burn: Add capability to find out what capabilities are available.
jib_: My proposal includes making it hard to use mandatory but also to prevent mandatory to fail.
fluffy: Not understanding the advantage of this.
dom: I thnk this discussion is not useful without a concrete proposal for handling mandatory constraints.
dom: Let's not base this on any assumptions we may have here.
dom: Depends on what approach we eventually take.
dom: So no assumptions should be made about what the mandatory constraints proposal will look like.
dom: We can handle extensibility by allowing the developer to know in advance what will be recognized by the browser.
dom: That we're at odds with WebIDL is not a good sign.
dom: But WebIDL is also a moving target with new approaches all the time.
dom: WebIDL is not the bible of Web API design just yet.
<Zakim> dom, you wanted to comment on WebIDL
+1 to jesup
jesup: Clear from the front/back camera discussion. Clear that there may be some constraints for which mandatory is never a good choice.
jesup: Need to consider whether _all_ constraints can be made mandatory.
[I'd claim that "camera direction" is never a good mandatory constraint to use]
jesup: Heard fluffy say he's ok with the idea of making mandatory constraints harder to use.
[and as such, should not be usable as a mandatory constraint]
jesup: So we define a well-understood way to do that.
jesup: In a way that doesn't confuse people.
[Travis suggests requiring using the control key to get mandatory constraints :) ]
jib_: That's exactly what I'm proposing.
fluffy: Chairs should set a date when we need requirements by on the list.
fluffy: Schedule some time for people to make proposals and for others to talk through those.
fluffy: Worried when we look at ways of doing that it may be dubious.
fluffy: So let's open it up but let's schedule that.
silvia: Want to point out a someone having used constraints in an app with 3 cameras...
silvia: well aware what mandatory means in that case.
silvia: but it's also important to be able to catch that problem.
silvia: making it hard for web developers to use it is not really a goal for me.
<jesup> I'll just say that I don't think we disagree on requirements, just on how to provide the result
silvia: I will find out how to use it anyway.
silvia: But most important is being able to catch errors and handle them.
jib_: I have a 3d camera. The app tells me 'I need to buy a 3d camera'. So I buy one. But it turns out the browser simply didn't recognize that.
silvia: I think we're discussing on different levels.
silvia: We need a proposal here.
<jesup> She's saying the error from failing mandatory should say "browser doesn't support 3d"
silvia: When we get an error on mandatory constraints failing we need to know why it failed.
burn: Hear what you're saying jib_ .
burn: Need to know if the constraint itself is known and supported by the browser.
[feature detection, as Travis mentioned]
burn: Can should be able to check which constraints are supported.
burn: what silvia said was there are different ways we can do that.
burn: check prior to call or handle as an error after invocation.
[a good summary of this discussion would include the list of requirements we have already identified today]
burn: would be interested in seeing proposals in both of those directions.
dom: This discussion has already identified requirements that any proposal in this space should have.
dom: Would be useful to conclude this 1st phase of this discussion.
dom: Are we aligned on the requirements we have identified today.
dom: Mandatory constraints should be less easy of less obvious than using optional constraints.
dom: Not all constraints should be useable in mandatory mode.
dom: Useful if developers can detect whether a given mandatory constraint is understood by the browser.
dom: Whenever a mandatory constraint cannot be fulfilled there needs to be sufficient information back the developer so they can adapt their code.
<scribe> ACTION: stefanh to send a call for requirements on mandatory constraints [recorded in http://www.w3.org/2013/11/14-mediacap-minutes.html#action03]
<trackbot> Created ACTION-27 - Send a call for requirements on mandatory constraints [on Stefan Håkansson - due 2013-11-21].
dom: I know some people already have ideas and requirements here. We should start writing down their ideas.
dom: Don't want to block on constraints for e.g. 6 more months.
dom: Let's write down requirements and share with the list.
stefanh: What's a reasonable timeframe?
stefanh: ok. 2 weeks.
adambe: no requirement on knowing when an optional constraint fails?
dom: There was no discussion on that. No.
dom: Mandatory constraints have been a hot topic for a while.
dom: There wasn't much clarity how and when those constraints should be used.
dom: So this is an effort to clarify those points.
stefanh: Encourage everyone to bring their requirements to the list.
dom: People that have ideas, write them down.
dom: ...related to existing use cases.
dom: I will be pushing back strongly on anything that requires a completely different API to the one we have now.
dom: But there is room for some small API changes.
<fluffy> if something is mandatory, and the browser can't do it, does the browser give you an error that the constraints can't be satisfied or just silently ignore the constraint.
<fluffy> if something is optional, and the browser can satisfy that, can the browser just ignore the constraint
gmandyam: Tobie's manifest on mobile dev. e.g. LinkedIn using browser sniffing to detect capabilities. This will happen.
gmandyam: Let's not think we can avoid that happening.
Travis: Looking at constrainable interface at the 'settings'. Is that defined anywhere?
burn: may be a problem with that link.
JimBarnett: settings is not defined Constrainable?
adambe: the link doesn't work due to a doc bug
Travis: So it's like a representation of one or more properties and their values?
JimBarnett: it represents the
current values of the properties
... defined in 10.1.3
<silvia> Settings needs an IDL spec
burn: Preference may be given ultimately to proposals that don't change our entire understanding of how the mechanisms work.
burn: How far that extends is a subjective call.
dom: Rephrasing: my interest is in shipping not just the spec.
dom: So what proposal will get this thing shipping sooner?
dom: If something is supported by browser vendors then that may carry more weight.
dom: Since that means we will be shipping sooner.
<fluffy> +1
burn: Lot's of descriptions how this thing should work. There are some expectations out there already on how this should work.
dom: If browser makers get pushback from developers then we may hear about it.
<burn> dan: not convinced that implementers always know best what developers expect
jib: are we going to discuss permissions?
burn: Would be better to discuss the permissions stuff first.
<jib_> jib: can we discuss permissions with gUM and the exploit of using mandatory constraints to play 20 questions to discover your cam info?
stefanh: Let's move on. Discuss permissions first.
burn: Which permissions are available at which point in the permissions check.
Travis: we should start with what the spec says.
burn: API method is now called getMediaDevices(...)
<fluffy> http://dev.w3.org/2011/webrtc/editor/getusermedia.html#methods-2
silvia: background on what chrome implements re: label attribute...
silvia: label is an empty string over http but exposed over https.
burn: suspect that's a consequence of being able to save http-based pages but not https-based pages.
fluffy: reasons we got to that is because we're trying to avoid fingerprint stuff.
fluffy: This all goes back to the fingerprinting and whether we can prevent that or not.
burn: With the way the spec is written right now - fingerprinting info can show up in the label attribute if you have accomplished a prior permissions check.
burn: Very important distinction about 'before permission granted' and 'after permission granted'.
silvia: In Chrome, I only get the label after permission check on HTTPS only.
burn: Believe this is the only info you can get.
<silvia> I ran MediaStreamTrack.getSources(function(result) {console.log(result);}); in the console of chrome 31
JimBarnett: You can get the capabilities of an object after permissions are granted.
<silvia> on both http and https I don't get labels
<silvia> I will only get labels after I have once been successful at getting a camera in https
burn: situation today is that you can only get the capabilities after you're acquired permission to access that device.
jib_: How about calling getUserMedia() with every single possible constraint. Then it fails and re-invoke gUM each time, removing one constraint at a time, until it works.
jib_: So I can figure out what the capabilities are before permission is granted.
dom: Another requirement for mandatory constraints may be 'do not leak information about the device'.
Travis: one use case I've seen requested is to avoid the permissions prompt at all since there's things you can discover from a constrainable interface
Travis: Is that a requirement we must satisfy or do we need to put permissions first.
jib_: What I proposed on the list is to remove the error for when something is overconstrained and instead always bring up the permission prompt.
dom: We need to go through the queue but you are well placed to bring up this as a requirement for more discussion on the list.
fluffy: We had previously come to the conclusion that the info available was slightly different depending on whether you had permission or not.
burn: we also realized that there is already so much fingerprinting info available already that this does not really add that much more info.
[I think the requirement we want to consider for mandatory constraints is "mandatory constraints must not enable to leak information about the hardware set up before the user has granted permission to one such device"]
burn: We want to close the gap with native and therefore developers need to know things before they start setting up the camera.
<jib_> why would you install a 3d app on a device without a 3d camera?
<jesup> Certainly is seems per previous conversation that's possible in the persistent permission case
<jesup> Teh question is can a non-persistent (or already-permitted) app can query that
silvia: About competing with native apps. They use up-front permissions. Ask once and then I get all permissions up-front.
silvia: Could we have something similar on e.g. first getUserMedia() request from a page?
silvia: If I'm on a dodgy web page, then perhaps that's the point I can leave.
dom: A lot of this is the browsers decision on how to enforce/obtain user's preferences.
dom: The spec can't say much on this other than this step requires user permission.
jib_: The Android permission model has been shown to be ineffective.
[this is a rathole; permissions on the Web is an unsolved issue]
jib_: No one reads it. The android model doesn't work.
jib_: Understand the use case of the web page that wants to know everything about the web page for full UX control.
jib_: But there's a reason we need to ask for use permission still.
jib_: Isn't the whole point of constraints that we don't need to get all the capabilities of the devices up-front?
<richt> <5 minutes away from the break>
fluffy: Certain argument to be made that Android and iOS permission models are the most successful ever.
<jib_> android and apple want the user's information, so yes = successful
fluffy: Lots of people would like to grant persistent, long-term permissions.
fluffy: I suspect the outcome of that discussion won't change from previous discussions.
silvia: There's a different between web sites I trust and web site that I don't.
silvia: And I make my decisions based on that.
dom: Lot's of other discussion elsewhere on this in W3C. Applies to things other than camera access also.
dom: My position: only limited information should be available without permission.
dom: question that has emerged is. does the mechanism for mandatory constraints have or not have the ability to protect this information sufficiently?
<fluffy> FWIW .. my view is it should not leak extra info if it does not have the info
<richt> <break. meeting resumes again at 15:30>
<scribe> Scribe: Travis
<scribe> ScribeNick: Travis
burn: We wanted to allow for
extensible constraints that don't require revving the
doc.
... lots of options
<burn> http://tools.ietf.org/html/draft-burnett-rtcweb-constraints-registry-04
burn: we have some constraints in
the doc for convenience
... It's not perfect.
... should survive ~5 months due to a recent update.
... we need to consider more constraints eventually (yep)
... don't want to have to get webRTC together just to update
constraints.
... I want to hear why IANA is not a good place
gmandyam: registry + expert review?
burn: You must define Policy +
registry
... currently "specification required with expert review"
... can't just way "good value"
<gmandyam> Sorry for jumping the queue - just wanted burn to clarify before we move on.
burn: need to get the expert to agree that the definition is sufficient
dom: worry about IANA is that
it's convenient for editors (of the spec) but hides from
impementors
... any process applied to the registry would be duplication of
W3C process
... process is there to help make sure the definitions make
sense.
... seems like the potential shortcut for extensibility doesn't
become a problem.
... not so familiar with IANA reg. proces,..
... but think about W3C specs as giant registry, etc. (in the
abstract)
... not pushing back very strongly, but...
... seems more like WebIDL dictionary keys or Enum, and could
look to make spec easier to update
<Zakim> dom, you wanted to ponder if the registration convenience is not hiding the true cost of adding new properties
dom: splitting these out into a separate process/community could cause more issues.
silvia: registries have been
talked about a lot in HTML.
... let's be aware of some of the problems and make a concous
decisions.
... 1) IANA may not move as quickly
... 2) may take longer to update than in reality
... (example of 3d camera constraint which gets implemented,
but lags in the IANA)
... in WHATWG they use Wiki approach
... open to public, public mentions it there.
... not advocating IANA or Wiki-based approach
... formal registries have slowness. There are some advantages
and disadvantages
fluffy: I will explain IANA
registries
... this is an IETF thing, and they can help
... can make a difficult approval process (one approach)
<dom> [but then isn't the registry a duplication of a W3C spec?]
fluffy: likely what they want is
a wiki-like registry (to ensure uniqueness of
names--basically)
... these are very quick.
... what we want--everyone provides a link to a doc w/explainer
and the result is a name
... our group can select names that are unique
burn: plugs for fluffy as an IANA
expert
... I've talked with other W3C groups about registries
... reason for IANA was that sometimes groups can last awhile,
but expertise can come and go.
... remaining connected via a registry is very lightweight.
burn: the thinking was that we'd get better response via a registry, than keeping a WG alive
gmandyam: for timely additions to
registry additions, seems like it's related to vender
prefixed
... we should go slow when a non-prefixed form (general use) is
proposed.
... we may need both slow and fast track in the registry
... I understood that expert review assured that this was
possible.
... not sure if expert review will work with IANA registry
burn: expert review can be done
however we want.
... we can relay instructions to the expert review to help
manage the process
<Zakim> dom, you wanted to comment on HTML registries comparison and to also comment on similar name-appropriation issues and solutions in w3c
burn: (can make it quick)
dom: registries in HTML are for
places where developers can innovate, not for stuff that
browsers depend on.
... meta names and rel values
... for registry of constraints, seems more like CSS
properties, which WG has been around a long time (to service
property requests)
... the CSS "expert review" gaurantees that they fit into the
CSS model
... vender prefixes have come out of vogue.
... now they are released behind opt-in flags
... my view is that our situation is more like CSS
... personal opinion: IANA registry doesn't add same level of
value as WG managed "list"
... what worries me the most:
... these are for interoperable constraints that browsers need
to ship--and related complications, bugs, etc.
... registry is so "easy" that we'll get unmanagable proposals,
no interop, etc.
... if that's the end result, then we fail our mission.
... yes, there's a cost, but it comes from the requirement to
get interop
burn: can you compare to a wiki?
dom: yes, great for feature
ideas, and open to the public
... but browsers don't need to ship these.
... some constraint (fake) is proposed, but browsers may never
ship it.
stefanh: same general thoughts as
Dom
... things to make mandatory for browsers shouldn't just be
"listed"
richt: Q:
... is it as simple as just coming up with a new
constraint?
... does our spec just handle that?
dom: This is kinda the problem I
was saying.
... in some cases in CSS you come with a variety of design
options. Assuming a constraint is exactly what you want may be
coming from the wrong perspective.
DanD: May have a chicken/egg problem
<richt> +1 dom
DanD: we could consider how much
volume of things we want to take on.
... how much value, discussions, etc.
... would be nice to know what the expected scale will be
... we really don't know yet.
... perhaps we can base our decision after we get more
traction
... maybe we could try a 2-step process
... there are options here.
... I see these as a fairly static collection of
properties.
... hope we don't add/remove every week.
<Zakim> silvia, you wanted to ask how long it typically takes to add something to a registry at IANA
DanD: if there's an early settlement of constraints, perhaps we start with a more collaborative process then change.
silvia: Seems like implementation
speed isn't much of a concern here.
... but rather, when 2 browsers implement, there is some formal
registration
... prior to that, a wiki may be enough (list of experimental
constraints)
... if two browsers agree then perhaps IANA kicks in, or the WG
looks at it.
... considering IANA: name added to list, and done
... if that's it then fine.
... if not, e.g., there may be side-effects, other
definitions...
... then it's not just a registry.
... it's a functional addition and registry is not
sufficient
ddavis: Q on testing
... would be be excluded from a W3C test suite if it was in a
registry?
<Zakim> ddavis, you wanted to ask about testing
dom: That's to my point--it should be part of the WG process
fluffy: may be confusing process of making a standard and registry list.
<dom> fluffy, but that's also what W3C specs are
fluffy: you can define what you
want the process
... it will never be the "required to implement" list
<burn> dom, the defined process for the registry can say to find or re-create a W3C group to discuss, if necessary
fluffy: may find that community invents new interesting things
<dom> sure, but then what does it bring on top of just saying nothing (i.e. new values get added by updating the spec)?
fluffy: want to clarify our range
of options.
... clarify
... more clear
dom: yes, a point to understand.
<Zakim> burn, you wanted to comment on SSML success because no group had to hang around all this time
burn: want to +1 fluffy
... it's _long lived_ (years and eons)
... process can define what to do with entries
... thinking about a group (that is gone), successfully left an
expert reviewer with the possibility of even re-starting a
WG.
... registry documents have processes to handle this sort of
thing (maintenance and what to do)
dom: Then why aren't we doing
IANA for CSS?
... since there's no need to create new entries in the
registry, then there doesn't seem to be a need to keep the
group around
... you can always ask W3C directly to re-start a group if you
want.
... seems like the same effect as having IANA redirect to
W3C.
... just not quite seeing the value
burn: left out the piece that if it's a small item, then it can be resolved rapidly through the expert.
dom: key question is mandatory
constraints
... browser vendors will want interop. An IANA registry
wouldn't likely be able resolve.
... others are just "optional" constraints (may not need)
... also
<Zakim> dom, you wanted to ask what Iana would add to W3C spec
dom: extensibility will come from
all angles.
... 3D cameras (when they come on the scene) will need much
more than a constraint def.
... may need much more for proper exposure to the web
burn: both processes support discussion in a working group.
dom: skeptical that the expert review being able to decide
fluffy: mandatory constraint doesn't mean mandatory to implement, just that browsers need to error when not supported.
dom: understand, but we don't have interop if they're not interoperable in the browser
fluffy: Let's talk CSS
... people want to move quickly.
... implementers just throw out css vendor prefixed stuff
willy-nilly
dom: CSS has learned valuable
lessons
... they've improved their processes to manage this.
fluffy: educate me
dom: I only have so much
experience here...
... but I've heard that browser experimental features will
operate behind a flag.
silvia: ... then as 2 implements do the same thing, then it becomes "official" and gets a standard
fluffy: Yep, seeing features
implemented quickly is great.
... when it takes too long people move to different working
groups.
dom: I see a need for flexibility
and extensibility
... quotes process as a means for making things happen.
<Zakim> silvia, you wanted to point out that we don't have to resolve this today
(lots of arguing)
silvia: if we think of IANA as a
wiki
... then it's overkill
... we have to deal with 2 standards processes
... agree with dom.
... if we just need an experimental name sandbox
... see the point for when the WG is dead
<dom> +1 to silvia
silvia: should be easy to add something to the tail of a doc in a registry
dom: Let's get this done. We can definately set up what we need to do.
silvia: some browsers may just start doing something and it becomes de-facto standard
DanD: could the list be used in
other context?
... just saying..
burn: Wrote a first draft. Not
sure what would go into the registry.
... lots of open issues
DanD: if it applies to more than just constraints, then seems valuable
ddavis: CSS works in modular
fashion. Lots of small specs
... seems like you could work the same way.
<dom> Travis: WebPerf is another example
<dom> ... they created an entire spec that defined a single property
<dom> ... it only took them 6 months to go to Rec
<dom> ... (hirestime)
<dom> ... there are also community groups
<dom> ... a variety of options to get things done quickly
burn: I see this living either
place. I've seen both ways.
... I tried very hard to keep things in W3C
... there was previous confusion about how things would work in
W3C
... just saying we should put it in the right place.
... if you're not familiar with IANA registries.
<dom> [I think I've argued why W3C seems like the right place, independently of IANA]
burn: then go learn about it.
fluffy: IANA is not part of IETF incase folks didn't know.
silvia: Wanted to suggset that
wiki page is a great idea no matter what the outcome
here.
... we need a place to germinate ideas.
... (for browsers)
... recommend putting a link into the spec for new feature
ideas
... we can then take it from there depending on what
happens
... seems OK to link to a wiki page from the spec (as long as
it's non-norm.)
dom: We also need to hear from
browsers.
... they know how much they are expected to innovate
silvia: Browsers can't see the future; this is for extensibility
<dom> Dom: it would be good to know what implementors think they see as likely need for extensibility and in what timeframes
<dom> ... clearly they have an important say on the need for this
stefanh: Let's do some agenda
bashing again...
... We had a discussion about lifecycle of media streams,
tracks, etc.
... largely agreement on changes proposed by adam
... then we discussed constraints
... there was consensus about needing them
... we didn't agree on all the requirements, but we saw a need
to review requirements
... there were some actions on the chairs, and some ideas are
floating around that should have more list discussion
... then a long discussion on the IANA registry
... determined that we don't need to decide it now
<gmandyam> OK
dom: per fluffy we need to figure
this out before LC
... we don't yet have concensus on the approach
richt: what happens if we do nothing and return to W3C for this discussion when extensibility actually happens or is needed?
dom: that means "go back to the WG"
silvia: expecting to close the group?
stefanh: it's a TF, may finish :-)
dom: what we are doing here is of
the same scale as CSS/HTML
... I expect a long lived group because there's a lot of work
yet to do.
... (sorry to disappoint)
<jesup> Can we set up something in the spec whereby if the group no longer exists it devolves to IANA? ;-)
gmandyam: parting thoughts--IANA's already in the spec and working? pundits can object via the list
<dom> ACTION: Dom to come back to media capture list on alternative proposal for registring constraints [recorded in http://www.w3.org/2013/11/14-mediacap-minutes.html#action04]
<trackbot> Created ACTION-28 - Come back to media capture list on alternative proposal for registring constraints [on Dominique Hazaël-Massieux - due 2013-11-21].
fluffy: dom please propose alternatives to list
stefanh: we didn't discuss
noaccess/identity
... this is an issue for LC
dom: from this morning's discussion, seems like noaccess/peerIdentity are poor fits for constraints
stefanh: Yes, agree.
... ready to push integration with web messaging to v2
... nothing to say about testing status (skipped)
burn_: on noaccess/peerIdentity (NO/PI)
<dom> Bug 22594 - noaccess / peerIdentity as constraints
burn_: important to consider
which portions of NO/PI belong in MCAS vs. WebRTC specs
... editorially, we have largely tried to avoid PeerConn. in
MCAS.
<dom> +1 to burn
burn_: we need to look at this
fluffy: Let's have the WebRTC
guys sort out.
... don't think we should go as far as removing
gmandyam: Argues that they slipped in when they shouldn't ahve
fluffy: Do we use the constraint
mechanism to handle these?
... but we know we're going to have this capability.
... we've always known
dom: Yep, we should sort out where these live, how they're integrated, and where they need to be defined
<gmandyam> There may be long-standing agreement to include a PeerIdentity mechanism in the WebRTC context, but not necessarily in the Media Capture context.
fluffy: WebRTC = martin + ekr
<dom> gmandyam, fair point as well, indeed
stefanh: LC for dec. 6 seems
unrealistic
... (since I have call to discuss requirements on
constraints)
burn_: Hey, we're making progress via these discussions
stefanh: when should we plan to finish
?
fluffy: The hard things: access,
permission, identity, etc.
... we are trying to land this spec with these hard
problems
<dom> +1 to fluffy on scoping down
fluffy: folks, go review the
spec
... does it have everything you think it needs?
... browsers are moving forward, they may not want to reset
(again, and again)
... what are the things that really need to change?
dom: +1
... we need to prioritize
stefanh: in WebRTC, we put out
the option to divide and conquer
... perhaps we can consider that?
dom: A complex problem.
... may be something for me and chairs to discuss
... clarifing...
... my philosophy w/ standards is what ships and has
interop
... good spec helps meet those goals
... we need to understand and build consensus on what is/will
ship and what is interoperable
... there are some non-interoperabilities
... we know mostly how to solve these.
... we should take input from everyone
... we collected it all and call it MCAS
... we need to continue thinking about the hard problems
... people seem to think about v2's as not happening...
... think more about it as modules that run in parallel.
... we run in cycles getting things to interop and moving on
and iterating.
... given where implementations are, the idea of not finishing
soon gives me despair
... it's a most difficult problem
burn_: I think we're much closer
than what fluffy said.
... I'm feeling encouraged as a result of various conversations
I've heard.
jesup: I also think we're
close
... part of discussing and working through the issues is
because folks have started to implement.
... we're now getting the implmentation feedback.
... we're moving in the right direction.
<dom> +1 to jesup
<stefanh> +1
jesup: we should be careful not
to open up too much...
... we're driving toward finalizing.
gmandyam: I have an image capture
presentation on the wiki.
... want to raise a few points before people leave (or fall
alseep)
<dom> Giri's slides
gmandyam: on slide 2
... To be useful for mobile devices, want to add focus and
zoom
... my problem without a registry, is that someone adds
zoom/focus and conflicts with what's in my spec
travis: browsers just won't go implement what's in a registry/wiki page.
gmandyam: 2nd thing: slide
4
... camera preview
... today we have kind=video or audio
... previews may be done off the same track, but there's
another idea.
... you could create a preview by frame grabbing.
... not ideal, but what FirefoxOS does--and doesn't like
it
... think I can propose a preview track.
... without constructors being a problem, I'm not as
scared.
jesup: can you detail the constructors issues.
?
gmandyam: harald proposed
removing the types/constructors
... will put together a proposal
adambe: is there a relationship to cloning?
<jesup> travis, I'm not sure i'm enough up on the issues. We only implemented MSTs relatively recently, and don't support constructors for them yet IIRC (jib_?)
jesup, I'm at a loss too.
?
Some discussion is going over my head
gmandyam: now on slide 6 -- going toward promises.
richt: how do feel about consistency with gUM
gmandyam: community seems in support of promises
fluffy: please take it to the
list.
... also, we may have made a mistake about promises