Generic Sensor API TPAC 2015 breakout

28 Oct 2015

See also: IRC log


Anssi_Kostiainen, Claes_Nilsson, Tobie Langel, and many others. About 50 people in the room
Tobie Langel


Tobie: Hi. Editor of the Generic Sensor API spec, for Intel.

Generic Sensor API

-> Generic Sensor API

Tobie: Lots of sensors being used nowadays.
... In mobile devices, now IoT. Lot of different sensors to expose. Goal of the project is to have a common base to expose sensors to developers.
... Sensor spec are often highly underspecified, which leads to interoperability issues.
... Also, exposing new sensors to the Web should not mean having to re-think the problem space again and again. Topics such as privacy and security should be common to all.
... These are the reasons why we're doing that.
... The spec itself does not specify any concrete interface.
... I'm looking at already published APIs to instantiate the spec. Also looked at other APIs such as the Automotive ones.
... Two main kinds of interaction with sensors:
... 1. continuous monitoring
... 2. change based on a threshold.

[showing example 3 from spec]

Tobie: The example shows the two use cases.
... Third case would be device orientation where you want to continuously know how the device is in space. One of the things that is missing from existing sensor specifications is better integration with hardware in terms of latency: low-level vs. high-level. The low-level concept is pretty useful in some use cases.

Louay: Support for multiple sensors of the same type?

Tobie: Totally. I forgot to mention that.

Louay: Can the browser do some discovery?

Tobie: Discovery is a very interesting problem. It very much depends on what kind of sensors you have. GPS chips would be local to the device, but as soon as you move to vehicles or WoT or Bluetooth networks, that's tough.
... Discovery is not in this version of the spec. We decided to tackle this in a future revision of the spec.
... There are on-going works such as the Bluetooth API.

Anssi: The spec provides an extension point.

Louay: The WoT IG discusses this, with the Thing API for instance. I have demos around that, come to the hall to see it.

ahaller: One of our works will be the Semantic Sensor ontology.
... How will you know what type of sensor you will be dealing with?

Anssi: It's about feature detection

Tobie: I have a section about that, but it's really about local sensors.
... There will be some information, not written in the spec. I want to make sure that concrete specs can extend that easily. I'm very much looking at the Generic Sensor API as a low-level building block that you can build on top of.

Nick_Doty: Thinking about privacy issues. Discovering sensors individually is useful but I'm curious how we're going to do that for generic sensors.

Tobie: A lot of sensors have privacy problems. Many of them are common to most of them.
... So it seems useful to tackle these common problems in the spec
... All of the boilerplate privacy/security concerns would be in the spec, while specific ones will be defined in concrete specs.
... Things such as naming sensors, exposing high-level or low-level sensors.
... When I started looking into it, I noticed that people kind of copy-pasted bits from other specs. I'm trying to rationalize this approach.

Kenneth: What is the timeline?

Riju: On the Chromium side, I'll start implementing next week. End of this year, I will release Ambient Light implementation.

Tobie: Right, that spec will be written on top of the Generic Sensor API.

Nick_Doty: other browser vendors interest?

Anssi: work started based on Mozilla feedback.
... We want to fix issues that arised in first device APIs now.
... I cannot speak on behalf of other browser vendors. Microsoft has it on its roadmap.
... So there's some commitment. Apple is the only missing block here, I think.

Shijun_Sun: I'm from Microsoft. Do you think there will be some impact from the Device APIs WG to other working groups?

Tobie: Yes, I can't speak for other groups but that's the goal.
... Geolocation would be a good candidate.

Shijun_Sun: It would be good to bring the topic for Media Capture, joint work between DAP and WebRTC.

<npdoty> Geolocation discussed device orientation this week

<npdoty> So I'm not sure we are all on the same page about where that api is going

Shijun_Sun: We've been looking at device rotations, tied to the camera

Tobie: If you have use cases for exposing that to developers, it would make a lot of sense to use the same API. Naming might be used to split ambient light sensors that are not linked to the camera from those that are in the device for instance.

<anssik> please feel free to open issues for topics raised at this session at https://github.com/w3c/sensors/issues

Tobie: You certainly don't think of getUserMedia in terms of a sensor thing, but the rest fits well, I think

Anssi: Please open an issue on the GitHub repo, people!

jyasskin: Do you have a polyfill?

Tobie: Yes, we played with a few specs, including the Geolocation API.

<npdoty> Whoa, are we going to hang off of window rather than navigator?

Tobie: There are polyfills. I should have a working demo, but broke it on my way here, sorry about that.
... I don't think we're going to be able to polyfill device orientation in a way that solves performance issues, obviously!
... I think I have a polyfill for the Battery API.

<npdoty> I think there is interest in an absoluteorientationevent in DeviceOrientation that might fit that use case

<npdoty> currently supported by Safari and interest from Chrome

Dave_Raggett: Wondering about the relation with the Streams API
... Electrocardiogram would be an example of streaming sensor.

<jyasskin> npdoty: IDL interfaces produce a window.InterfaceName. One can suppress that, but not reparent it.

Tobie: If you look at async primitive space, you have Promises for one-off events, then you have Observable for a bunch of events. The sensor space we're dealing with sits somewhere in the middle.
... I think it would be a mistake to not make the distinction between streams and discrete events. Our mental model is different.

Dave_Raggett: heartbeat is just an example. You could have regular different measurements and other things.

<Mek> jyasskin: that doesn't mean it has to be window.sensors as opposed to navigator.sensors (but yeah, all the types would still pollute global scope)

Tobie: Two main kinds of sensors: precise interval measurement, and those where you're really interested in the notion of change.

<jyasskin> Mek: Yep. I think Tobie said he's dropped the foo.sensors namespace entirely.

Tobie: Most sensors pull the data out, and then pretend to push it when a threshold is met.
... Heartbeat example is interesting because it's a high-level sensor that could be built on top of lower-level sensors.
... The data you get out is a computation of other types of sensor data.

Mikko_Terho: Confused about high-level / low-level

Tobie: For the purpose of the spec, those that are tied to an implementation, we call them low-levels.

Dave_Raggett: There are different families of sensors in terms of patterns.
... In the IoT space, lots of people think in terms of reading a value. Not really thinking in terms of streams. There is perhaps a taxonomy of different sensor types independent of the implementation.

<npdoty> For Geolocation, we consider it a feature, rather than a bug, of having a datatype agnostic to the particular mechanism to determine it

<npdoty> Is it useful to define a lot of low level sensors if we don't expect them to be used or for users to understand them?

Mikko_Terho: For high-level sensors, you'll get a callback, basically. At the low level, you do the monitoring. My recommendation is that the high-level API output chould be changes.
... Then if you want to monitor a complete stream, you need to change the model.

Riju: we're not considering cameras as sensors per se.

Dave_Raggett: Then there's some sort of assumption of what sensor types are.
... Have you considered the notion of timestamp?

Tobie: Every reading is timestamped with a high-res timestamp.
... Based on the local clock.

Claes: Have you considered to use the Streams API in the end, for a future version perhaps?

Tobie: If you need a stream, I think you do not need something like the Sensor API.
... I'm thinking about 1Khz sensors, roughly.

Claes: The Streams API is a good starting point. Lots of functionality from it could be reused. I think it should be considered.

Dave_Raggett: Maybe the assumptions should be made explicit at the beginning of the spec.

Tobie: OK, I'm happy to look into it, e.g. for the device orientation API.

Anssi: I think Claes is looking at it from the perspective of raw sockets.

Tobie: Could you put an issue on the GitHub?

Claes: Yes.

Adam_Alfar: [missed comment]

Mikko_Terho: Also need a way to agree on what constitutes a change betwee browser and sensor

Tobie: Links back to discussion on threshold we had yesterday.

Mikko_Terho: It goes beyond that. I might be only be interested in 5 degree changes in temperature for instance.

Tobie: Right, the API addresses some use cases but not all of them.

Mikko_Terho: I would differentiate these 2 cases

<npdoty> Yeah, there seems like an efficiency and privacy/minimization advantage to letting the site request the level of detail it needs

Riju: This API gives something to developers that is not exposed by other concrete APIs: the ability to set the frequency with which you get readings.

Dave_Raggett: Continuously-changing variables can be approximated with discrete values. Have you discussed this?

Tobie: No.

Anssi: I think we don't have concrete sensor products in mind for this.

Dave_Raggett: You could measure a sound, for instance.

Tobie: That's where I would say that this out of scope.
... Would you be willing to open an issue?

<anssik> please feel free to open issues for topics raised at this session at https://github.com/w3c/sensors/issues

Dave_Raggett: Yes.

[some discussion on the topic]

<riju> Dave for scope, also look at https://github.com/w3c/sensors/issues/1 and comment there

Shinju_Sun: Do you think it would be useful to give developers the accuracy of the measurement? For some sensors, the accuracy may depend on the value, fluctuating.

Tobie: Yes, accuracy is in.
... specific to each measurement.
... already in the draft.

Shinju_Sun: I noticed you're using Promise already. And also talk about low latency. What do you consider low latency?

Tobie: The API is EventTarget so you're listening to callbacks. Promises are used as a convenience method for one-off readings.
... One of the things I want to do for performance is to see how this can be tied to requestAnimationFrame
... The most critical requirements I've seen are linked to virtual reality, not to throw up ;)
... If it's combined with animation frame rates and WebGL, it should be doable to have the right latency.
... I'm trying to figure out how to do that in the spec. I don't even know if it's doable from an implementation perspective.

Virginie_Galindo: I'm from Gemalto. There are some concerns about sensor data connections.
... There were some new models starting to be discussed: you may want to read data from sensors without anyone listening to that.
... Have you thought about transferring sensor data in a cyphered way?
... Has it been discussed?

DKA: That's what I was going to ask.

Tobie: I'm not sure I understand the question properly.

Virginie_Galindo: The question is about indicating whether the reading was transfered securely.
... You need to have secret keys. You may want to convey data that is conveyed in clear and data that is not in clear.

Tobie: Would that be available to the Web application?

Virginie_Galindo: I'm really thinking about the future. Please think about that.

Anssi: I think we ran the spec to the security group for review already.

Tobie: I want to understand the use cases more precisely.

Nick_Doty: you want encrypted values coming from the sensors that the browser would not be able to decrypt?

Virginie_Galindo: One use case is that of a sensor provides cyphered values, not readable by the Web app (for privacy reason). Second use case is aggregation several data readings by the Web app and encryption by the Web app afterwards.

Kenneth: you may want to send the data without reading it to a server.

Tobie: A kind of opaque reading is what you need.

DKA: Related to permissions and fingerprinting and stuff like that. Regarding permissions, in the Powerful features spec, it talks about knowledge of external devices as being a powerful feature.

Francois: I think that's already in the spec

Tobie: Yes.
... Permission is an implementation detail. You just scope that out in the spec.

DKA: The spec needs to be clear about that though.

Tobie: Right, the spec will have a section that is larger than the title it has right now.
... There is HTTPS concerns, so it's pulled that as a requirement in that spec.

DKA: I would also encourage you to take a look at fingerprinting.

Tobie: I started to read that on my way here. Great stuff (altough this broke my laptop).

ahaller2: Fingerprinting may be already a lost case.

DKA: Discussed on the TAG.

<dka> Unsanctioned tracking: http://www.w3.org/2001/tag/doc/unsanctioned-tracking/

Dave_Raggett: Detecting when a Web site looks suspicious. Can this be detected? Could the spec say something about that? Third-party apps could perhaps monitor sensors somehow.

Nick_Doty: Researchers have done that in a way that instruments the browser.

Tobie: You really want to handle that somewhere else.

Dave_Raggett: Yes but that could be called out in a Privacy section.

Johannes: Can the value be opaque to this API? Does the API really have to understand what it's dealing with?

Tobie: Does the operating system or user agent need to understand the value?
... This is the usual security vs. performance issue question.

<npdoty> I think there is a privacy advantage to the fusion approach as well, potentially

Tobie: You can build a pedometer using data from a gyroscope. You have to read it at something like 60Hz. You're draining the battery. Now if you let the underlying system do that, it may be able to do sensor fusion and expose a pedometer, thus improving performance and saving battery.
... The more you move to the sensor, the better the results in terms of performance.
... I don't really have good use cases in mind for opaque readings. For the use cases that we've looked at that are around non-opaque readings, it's better to move closer to the sensor.

Louay: About connectivity issues with remote sensors. Do we need a feature in the Generic Sensor API?

Tobie: Yes, probably some error events. Part of handling the error could be to create a new sensor again to talk to the sensor again. I don't expect the sensor to survive disconnection issues for the time being.

[breakout session adjourned]

Summary of Action Items

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.140 (CVS log)
$Date: 2015/11/03 11:07:31 $