Media Capture Task Force Teleconference

<trackbot> Date: 25 June 2014

<stefanh> draft agenda: http://lists.w3.org/Archives/Public/public-media-capture/2014Jun/0118.html

fluffy: I would like to note for the record we have very few people here to make decisions, we should be careful

hta: W3C doesn't impose mailing list decisions the same way the IETF does
... we have to make decisions, and we'll notify the list

fluffy: I object to make decisions given the current attendance

burn: hta is correct that we're entitled to make decisions given advance notice given etc
... but indeed, we're not many here, and the chairs might want to open some (potentially contentious) decisions on a case by case basis to the list

<fjh> you could send a Call for Consensus, CfC on the list, after rough agreement on the call

Frederick: note that we can reach rough decision on the call, and follow with mail-based call for consensus

Approve minutes from last meeting

<stefanh> http://lists.w3.org/Archives/Public/public-media-capture/2014May/0264.html

RESOLUTION: http://lists.w3.org/Archives/Public/public-media-capture/2014May/0264.html approved as minutes

"Ideal" algorithm

Stefanh: let's plan on 20-30 minutes on this topic

<jib> https://docs.google.com/presentation/d/1jTz3pfEmgXn3y6Mk-s2SZaq3wa0VBEI7QnTR05JVR3s/edit?usp=sharing

jib: several options on the meaning of "ideal"
... a pure hint
... a fully algorithmically defined feature
... a shortcut for a set of "advanced" constraints
... 1 and 2 are not very appealing to me
... I have a "minimum" algorithm proposal for 3
... [reading from slide]

hta: could you give more details on not providing new functionality?

jib: dan was concerned about testability
... if this is just a shortcut, then we have testability (?)

fluffy: no comments on this; my question was at a more general level
... a broader way to solve the general problem
... I have no issue with Jan-Ivar's proposal
... "ideal" is a little bit complicated in the general sense
... if it's not deterministic, it's not useful to anyone
... JIB's proposal fits that requirement
... inside the deterministic behavior: say a constraint where device ratio is required to be 1
... and ideal width set to 400, ideal height to 600
... we need to make a choice between global optimum vs local optimum
... with a global optimum, you would get e.g. 500 for this case
... the other approach is to pick one property to optimize for first, e.g. width
... in which case you get width and height to 500
... for the use cases I can think of, local greedy algorithms seem a better match
... and J-I proposal fits there
... but for them to be deterministic, we need an order
... since we've renounced to let authors set that order, we should set a default order
... for instance, based on the alphabetical order of their names
... once we have that, we need to determine how to determine the optimal value
... could be default metrics, but there are other models, e.g. squared metrics
... is this making sense?

burn: yes; note that metrics need to be nuanced with e.g. binary values

fluffy: my proposal is to define some default metrics, but open the way for different metrics for different constraints
... in particular, some constraints might have enumerated values where there is a logical order
... (e.g. low / medium / high)
... So the greedy ones are easier, the global ones more powerful, but not clear whether we need the power of global
... but given that the ideal supporters are not around, I'm a bit worried we will get a biased decision
... then we need to define our metrics, and define the order on which these metrics are built

hta: the algorithm that J-I noted, the way you add the various constraints changes from local to global

fluffy: I think we can find ways to specific this concisely; I'm more worried about what to specify

burn: I want to raise a question about how deterministic we want this algorithm to be
... J-I proposal is not as deterministic as what Cullen is asking for

fluffy: J-I, do you mean the algorithm to be deterministic?

jib: no, I don't
... @@@ the discussion by saying "no additional features"
... if we say everything is deterministic, there is no room for UA discretion
... we already have "advanced" for a complete deterministic approach
... "ideal" is for people that don't have that level of requirements
... and are happy for the UA to pick reasonable values

<Cow_woC> Why do we need UA discretion? The only point of the current approach (as I understand it) is to avoid fingerprinting. You don't need UA discretion for that.

fluffy: if you want to say "the largest video possible", you can in theory get it with advanced, in practice, it's not practical
... a deterministic algorithm doesn't imply no room for distinct behavior between UA

jib: I don't think I understand that

fluffy: I meant deterministic as the user should be clear on what to expect
... the same way the "random" function is deterministic

<hta> there are three things being talked about - "all browsers behave the same", "we set up common user expectations, but browsers may differ", "the spec is silent".

burn: my main concern is getting testability (with some range of testable)

fluffy: the main point is that if the constraints ask for "ideal width" of 50, "ideal height" of 50, and the device is capable of it, the browser must give that

burn: I agree

jib: I agree too
... getting "ideal" implemented with the existing algorithm makes it easier to implement

hta: I hear rough agreement on J-I proposal, with some additional expectations on local/global

stefanh: also questions around metrics

fluffy: J-I's proposal talks about "range center" — that implies linear metrics, and doesn't work for non-numeric values

burn: I think to make progress on this is to provide a list of constraints with ideal, combined with scenarios of what the underlying browser is actually capable of
... and see what would be reasonable outcomes and then evaluate the algorithm

fluffy: getting back to my earlier example…

hta: that example is actually a user that can't make up his mind?

fluffy: not necessarily, imagine a scenario where the window gets resized
... as hta noted, depending on how you construct the ideal stack, you get either a local or global optimum

jib: not sure we need that level of details given that advanced already provides that

fluffy: I don't think I can get that via advanced with reasonable performances

jib: but browsers can compete on what "ideal" means, from which we can learn
... we can do that by leaving it up to the UA to determine which order they follow, which steps they'll increment in their comparisons, etc

ekr: not sticking to "APIs should have deterministic outcomes" makes me sad
... this seems a sad outcome of avoid fingerprinting

<ekr> dom: APIs should have deterministic outcomes.

<ekr> Not having htem makies me sad

jib: my main concern is to get done with this, ideally without too much work

hta: we should avoid surprising the users, e.g. when an outcome ends very far from one of the "ideal" constraints
... the headache is to find which algorithm can be realistically developed for that

fluffy: your earlier proposal seemed to accomplish that

hta: if you make reasonable assumption on incremental steps, order of constraints, etc
... that makes a lot of things to specify

jib: if we specify alphabetic order, and an increment step, we're good?

fluffy: I think so; and of course, browsers can follow a different algorithm as long as it gives the same result

burn: "ideal" is meant to be simple; the minimum requirement is that if the browser can provide a configuration that matches all the ideal, it should
... if it can't, fluffy's algorithm seems reasonable, but there can be other reasonable approaches

ekr: we're talking overcomplicated algorithms only to avoid property enumeration of devices
... I think we should revisit this decision instead

burn: it's not that simple; there is the case of multiple concurrent usage of a given device where you want the UA to have freedom
... likewise for transmission on peerconnection

ekr: to solve concurrent requests, you could simply say "the last one to ask is has the least say in what he gets"
... I think getting weird behavior from other apps using the cameras
... is not acceptable

burn: the spec has been defined that way since day one
... I think we're going into a rathole about defining a close approximation of what "ideal" gives you when it can't give you a match

<jib> https://docs.google.com/presentation/d/1vtdzuPUCeIHHyW7uhIrL6TNCZZHHd0tSelX61dgw6R4/edit?usp=sharing

jib: so if we were to remove "ideal"
... we could back to the "mandatory" heading syntax
... as long as we have the list of supported capabilities, I'm fine with that
... this gets us back close to what we had earlier

ShijunSun: we like the "ideal" definition as a hint
... different devices might have different capabilities and settings
... depending on the scenarios (e.g. local or rtc), the UA will adapt
... it's difficult to define how close the UA should target
... since that depends on the devices and scenarios
... "advanced" is already there for deterministic precise behavior
... I think we should stick to "ideal" being simple

fluffy: none of the algorithms under discussion allow for adaptation under changing circumstances

[+1 to ShijunSun FWIW]

fluffy: "ideal" is an optimization problem
... it looks useful, but we may have to reconsider it if we can't specify it

<Cow_woC> An API that leads to unreliable (non-deterministic) results will not be used for anything except "Hello World".

<burn> I think fluffy above said that none of the algorithms under discussion prevents adaptation under changing circumstances

hta: we have promising discussions, but no consensus, and lack of input from the "ideal" supporters

fluffy: I see a range of proposals, and can imagine their implementation impact
... I don't really care about greedy or global
... we need to base our decision on requirements
... I care for it to be deterministic
... which then gets us into ordering and metrics considerations
... once we have these, I think the algorithm should be easy to build
... I heard ekr say we should forget about the whole thing and fingerprinting

stefanh: would you be willing to write this up as a proposal?

fluffy: J-I, are you interested to work with me on turning your proposal into a global proposal?

jib: sure

<scribe> ACTION: Cullen to work with JIB on a proposal for "ideal" [recorded in http://www.w3.org/2014/06/25-mediacap-minutes.html#action01]

<trackbot> Created ACTION-29 - Work with jib on a proposal for "ideal" [on Cullen Jennings - due 2014-07-02].

"how long do permissions last"

hta: I think we have a clear picture of what people want now

<stefanh> Harald's slides: http://lists.w3.org/Archives/Public/public-media-capture/2014Jun/att-0137/June_Media_Capture_Bug_Walkthrough.pdf

hta: single use permissions last until all tracks source from a given device stop

<ekr> This seems satisfactory to me

hta: device names are available until all devices have stopped
... stored permissions last until revoked

<ekr> With the caveat that obviously if we were to go to direct access, then this last bullet would be irrelevant

fluffy: I'm not sure how "closed" ties to what the user will perceive

hta: permission goes away when javascript closes the stream or user revokes it

ekr: the user perception question ties to the "indicator" question
... we have two indicators ("can tap" and "is tapping")

fluffy: I think the permission should be linked to indicators

ekr: they are linked by the "can be used" indicator
... the question is what makes you lose the ability to use a capture device?
... and that's by the user muting the mike or the app closing the stream

hta: this is a matter of distinguish the cause and the effect

adambe: re access to the labels
... I sent a mail on this
... there is not much point in revoking access to data once it's been granted, since it can be stored separately once it has been obtained
... although we should "hide" new information until permission is granted again

hta: you're right, this is specifically about new devices

ekr: agree that there is no much sense in protecting names that were already shared

<ekr> oh, sorry, didn't realize we ahd a queue

ShijunSun: from a security and privacy perspective, any part of the page (incl. iframes) can trigger getUserMedia and thus the permission UI

burn: just before the call, I've reviewed all the emails linked to permissions in the last month or so
... what is the outcome on the "reload" issue?

hta: since we're tying the permission to the survival of JavaScript objects, we don't survive reload

burn: I'm happy with that, but just wanted to clarify it in the context of this decision

ShijunSun: just to be clear, there is distinction between "suspend" (e.g. on mobile devices) and "reload"
... we should survive suspend

hta: yes, as other JS objects

burn: and that also matches my expectations as a user

RESOLUTION: bug 22214 resolved as per the proposal (making sure the text matches this)

Bug walk through

hta: bug 25248: stop() method and "ended" event
... some APIs do as we currently do (don't signal your own actions)
... but I like the consistency of always firing

adambe: I prefer consistency with other APIs in the platform
... for stop(), it's no big deal, but for addTrack and removeTrack, it gets trickier
... I think we should keep it the way it is

ShijunSun: I think always firing better, since it allows better layering of libraries

burn: so the question is whether "ended" fires only when a stream stops outside of script control, or fires in all cases of stopping a track
... so the question is what will get done in the "ended" handler
... and it seems consistent to handle all "ended" events consistently
... (whereas adambe's example of addTrack might follow a different model)

dom: I think we should go as other APIs do
... and if we hear clear dev feedback that this is bad, we can always revise it
... consistent eventing of "ended" can be dealt in user land in any case

<ekr> If we're voting, I vote against firing

hta: I'm hearing more arguments for not firing than firing, and no particularly strong argument for firing
... so I suggest we keep the spec as is
... will take the remaining bugs to the list
... Please note the "How You Can Help" slide: in particular, clear bug reports, and pull requests to fix them

fluffy: for controversial topics, avoid bugzilla and pull requests, prefer the list
... for clear mistakes, pull requests are perfect