W3C

Device and Sensors Working Group Teleconference

20 Sep 2016

Agenda

Attendees

Present
(no, one), Anssi_Kostiainen, Tobie_Langel, Tomoyuki_Shimizu, Kenneth_Christiansen, hadleybeeman, Wanming_Lin, Koichi_Takagi, Dominique_Hazael-Massieux, Rijubrata_Bhaumik, Jonathan_Jeon, Maryam_Mehrnezhad, Harald, Alvestrand, Kaz_Ashimura(W3C), Lukasz_Olejnik
Regrets
Chair
Anssi
Scribe
dom, anssik, tobie

Contents


<tobie> kenneth_: "there is something weird about the naming of specs - in some case we name the thing measured, some time the things that is measuring" That's by design. Low-level sensor: thing that is measuring; high-level sensor: thing that's measured.

<hadleybeeman> trackbot, start meeting

<JonathanJ> #webpayment

<inserted> ScribeNick: dom

Agenda review

Anssi: [explaining the layering of sensor APIs, where concrete sensors fit and how fusion would work]

@@@: the issue with doing fusion API in JavaScript is that it will be much slower than when done in hardware

scribe: you probably need both low and high level APIs

Hardley: what motivated the work in this space?

Tobie: my approach has been if something has been exposed in Native APIs, we probably need to consider it fro the Web as well

Hadley: any intersection with the geofencing work?

Dom: the geofencing API has stopped, but one of the issues that stopped them was around permission for background processing
... which we will have to consider for sensors

Tobie: there will be 3 layers of sensor usage: main thread, worker, and service worker
... the latter is the one with the most difficult aspect from a permission perspective

Generic sensor implementation feedback

Riju: we have been working on implementing the sensor API in chromium
... I hope that within the next month all of our work will go upstream
... we have been working on ambient light as the 1st API to land
... acceleratomer, gyroscope and magnetometer are the next ones
... we are also asking the WebVR guys if they're interested in proximity

<tobie> Intel implementation feedback blog post: https://01.org/chromium/blogs/riju/2016/generic-sensor-api-javascript-powered-platforms

Riju: Before shipping, we need to sort out the permissioning story which is still pending

Kenneth: [showing demo]

<riju> https://01.org/chromium/blogs/riju/2016/generic-sensor-api-javascript-powered-platforms

<riju> the last part has a link to the chrome apk, demo, we will add the apk soon

Kenneth: [compass demo based on the new sensors API]

maryam: what does the new API brings over the existing device orientation API?

riju: you get to pick the frequency (instead of having it hardcoded in the UA)

maryam: but the UA could still hardcode the possible frequency

tobie: sure; but for instance, you could limit the frequency based on permissioning

riju: right now, the device orientation API is not under any permission protection

maryam: have you considered hybrid apps?

tobie: a hybrid app will inherit whatever the Web view provides

dom: in general, hybrid apps are not directly in scope for W3C
... they are impacted by our work as a side-effect of our work

riju: the demo works on Chrome on Android, but also on Chrome OS, Linux, MacOS

Anssi: there has also been implementation work in constrained environments

<maryammjd> some stuff is mentioned about hybrid apps here: https://arxiv.org/abs/1605.05549

<zkis> https://github.com/zolkis/iot-js-api/blob/board-io-ble/api/sensors/README.md

zkis: we do hardware API in JavaScript; we've been looking at how far we can go with the W3C API in a constrained environment
... some of the changes we've made are that we list the available sensors on the platform
... which might be part of a future version of the API
... also need to document controller and sensor name

Tobie: this looks a lot like some of the work I had done when there was discovery in the API

zkis: for discovery, you need to have a namespace for your sensors

@@@: are you actually using WebIDL in this constrained environment?

zkis: no, it's just for illustration purposes

@@@: I've been wondering if there should be an IoT-compatible WebIDL

Tobie: that would be a massive amount of work
... I'll be looking at the intersection of WebIDL & Web Assembly which may cover this

<kotakagi> CHIRIMEN presentation --> https://drive.google.com/file/d/0B6BpMxqdTZPCQWRHR0EtSVBrUDQ/view?usp=sharing

Anssi: onto the KDDI demo
... is this a collaboration with Mozilla?

kotakagi: it's research at this point with not only Mozilla but people in CHIRIMEN open hardware community; we're doing this in a Community Group

Anssi: are you considering adopting the generic sensors API?

kotakagi: that's part of the things we're looking at
... The CG is browserobo

<Tomoyuki> https://www.w3.org/community/browserobo/

Generic Sensor Level 2 issues

Anssi: some of the issues we might want to move to level 1

<dom_scribe> Generic Sensor Level 2 issues

Tobie: Looking at issue #95

riju: not sure if it's still needed with the state change event

<anssik> [the network was down, so we we took offline minutes for the last hour]

<anssik> wanming, feel free to share your ambient-light test suite status on IRC

<wanming> ok

<wanming> have you seen my mail sent to the mailling list yesterday?

<anssik> https://lists.w3.org/Archives/Public/public-device-apis/2016Sep/0019.html

Tobie: issue #94
... cancelable promises
... TAG feedback was not to wait for them
... so we should move that issue to level 1 to return a (non-cancellable) promise
... next, need for calibrationneeded event?
... the use case for that event is to replace the default "browser calibration request" with a local app UI
... needed for stuff like magnetometer
... so probably needs to move up to Level 1
... or maybe a "nice to have"

Anssi: is that implemented in deviceorientation?

Kenneth: never seen it in a browser

Riju: just the checked the code in Chromium, and haven't found anything on calibration

Anssi: I guess maybe level 2 is appropriate

Tobie: Issue #89 and #88
... one-shot will need to move to level 1
... cached values is part of Geo

Anssi: Geo is an interesting spec to compare ourselves with
... We may need to have the conversation with the Geo WG

Dom: I think a good path forward would be to start incubating a generic-sensor based geo API

Anssi: sounds like a good plan
... #88 and #89 ought to be level 1 as a result
... #63?

Tobie: #63 needs to be closed from what I recall
... one of the things we wanted to do is to expose the actual frequency the sensor is running at
... the reason this is not exposed is that OS APIs do not expose it
... Hmm looking at the details now, I think the issue needs to be reworded
... definitely a level 2
... the implementations aspects are challenging on some platforms

Anssi: issue #62?

Tobie: important e.g. for gyroscope, if you want quaternions rather than tait-bryan angles
... since it applies to multiple specs, I think it needs to be in the generic sensor API
... this is about how the data is exposed to the Web devs

Dom: my recollection is that for instance using quaternions gives you more stable data in some orientations, which means that the right data representation is use-case dependent
... so we should provide a way to make that decision

Kenneth: there are cases where you may need various data representations for the same sensor
... I'm not sure that invoking different constructors will be well-perceived by developers

Hadley: as a developer, predictable data is great
... if each of these models is specific to each concrete sensor
... why does that need to be in the generic sensor?

Dom: to get a consistent unit-selection mechanism across sensor types
... FWIW, I think "unit" is too specific; in general, it's a question of data model

zkis: it's usually refered to as "representation" in the sensors world

Anssi: it's not looking like we're getting consensus

Tobie: I'll make a proposal and people should feel free to disagree with me

Maryam: many sensor APIs use common names across platforms
... we should try to align with those

Tobie: I've found lots of variation across platforms
... but that's not really a generic sensor API problem

Anssi: let's park this for now

Tobie: I think we need to move to level 1 and I'll make a proposal

zkis: I think the current proposal is suboptimal
... we should use observers for this

Tobie: I'll make a proposal

Anssi: issue #12?

Tobie: I think we need to make workers-availability part of level 1

<anssik> ambient-light#5.2.1. The AmbientLightSensorReading constructor

Anssi: I've re-labeled it

Kenneth: I would like to better understand the use cases
... because the serialization costs of workers will kill many high-frequence use cases
... not sure where main-thread sensors will be causing jank

Anssi: what's the implementation cost of bringing it to worker?
... so that we could have an experimental build to evaluate the performance value of having it worker

Tobie: I think the use cases story should be discussed part of the TAG review discussion

Kenneth: will do

<scribe> ACTION: Kenneth to discuss with the TAG why workers are needed [recorded in http://www.w3.org/2016/09/20-dap-minutes.html#action01]

<trackbot> Error finding 'Kenneth'. You can review and register nicknames at <http://www.w3.org/2009/dap/track/users>.

Ambient Light test suite

<anssik> https://github.com/w3c/sensors/issues/134

<dom_scribe> Report on ambient light test suite

<anssik> https://github.com/w3c/sensors/issues/135

<anssik> thanks wanming for the test suite status, we'll now move to the next agenda item

<wanming> thank you for the time, I am going to drop the meeting. :)

<anssik> https://github.com/w3c/sensors/issues/131

<scribe> ScribeNick: anssik

spec has guidance on naming https://w3c.github.io/sensors/#naming

low-level sensor comes without "Sensor" postfix

current concrete specs do not obey this convention

is this a good convention to follow?

<maryammjd> https://developer.android.com/guide/topics/sensors/sensors_environment.html#sensors-using-temp

anssik: anyone disagree with the naming guidance https://w3c.github.io/sensors/#naming

[hear no concerns in aligning naming per https://w3c.github.io/sensors/#naming]

discussing good new new names for interfaces

came up with a list of proposed names https://github.com/w3c/sensors/issues/131#issuecomment-248268461

[triaging https://github.com/w3c/sensors/issues?utf8=%E2%9C%93&q=is%3Aissue%20is%3Aopen%20no%3Amilestone]

[tagging issues, adding milestones]

https://github.com/w3c/sensors/issues/127

tobie: no agreement on namespacing (yet)

https://github.com/w3c/sensors/issues/126

tobie: add a note to the spec, that this is implementation specific

dom: is there no reasonable default per sensor type?

tobie: if reporting mode is periodic, it is periodic because a frequency is requested
... for light it does not matter

https://w3c.github.io/sensors/#construct-sensor-object

If frequency is not null,

If sensor supports periodic reporting mode,

Set sensor_instance’s desired frequency to frequency.

Otherwise,

throw a TypeError.

tobie: should the API report an error if frequency is set if the reporting more is auto?

<mikhail> on construction stage we do not know the actual reporting mode yet

<mikhail> reporting mode is fetched from platform so an async call is needed

<mikhail> i.e. start + onchange event

tobie: agree with mikhail

<mikhail> and similar no 'default sensor' notion is available at sensor construction

[tobie adding a comment to issue #126]

tobie: made this a Level 1 issue to be fixed soon
... issue updated https://github.com/w3c/sensors/issues/126#issuecomment-248274234

do,m: getting confused, spec says the reporting mode of a concrete spec

d/dom/dom/

tobie: we're backing out given new information that emerged

<riju> so auto == onchange and periodic==continuos

tobie: specifically, because a concrete sensor specs we have for now do not support reporting mode switching, so we actually know up front at construction time what is the reporting mode
... every spec defined its reporting mode

<mikhail> question: why do actually need to mention 'periodic mode' in the spec?

<mikhail> what benefit for user?

tobie: which user? web developer?

<mikhail> same sensor type can have implementations (on different platfroms) with different actual periodic modes

<mikhail> yep, we developer

<mikhail> webb developer

<mikhail> and again this is something we don't know at construction stage

<mikhail> can we always treat the input frequency as a 'desired polling frequency' hint?

tobie: looking at all sensors exiting today, only periodic sensors are gyro, accel, magnetometer
... all main use cases for these are to poll periodically

dom: my suggestion, simplify to do not allow multiple reporting modes by sensor type

anssik: noting accelerometer, gyro, magnetomer are currently spec'd as reporting mode "auto", need to fix

<shalamov> on Android, Mac and linux (industrial io) platforms we also poll periodically for each sensor, but depending on the sensor type, we notify developer everytime we get new reading from HW (periodic) or when value is changed from previous reading (on change).

<shalamov> Who should decice what mode is supported by the sensor? Specification or platform?

<dom> right now, the spec decides

<shalamov> Then the question, should onchange event be generated for sensors with periodic mode?

<dom> shalamov, yes

<dom> (onchange signals the reading object has changed, not necessarily its attached value)

<mikhail> what is the difference from the web developer perspective if one or the other periodic mode is assigned to the sensor?

<dom> from the web dev perspective, with a "periodic" sensor, it can set (or at least request) a specific frequency

<dom> not with an "auto" sensor

Change reporting mode to "periodic" https://github.com/w3c/accelerometer/issues/6 https://github.com/w3c/magnetometer/issues/13 https://github.com/w3c/gyroscope/issues/5

<riju> spec periodic == android continuous, spec auto == android onchange+continuous

<riju> got this answer from Tobie

<shalamov> Sensors with auto reporting mode (onchange) do require frequency, so that hw will poll sensor at specified frequency and notify whenever reading is changed, e.g. AmbientLightSensor.

<mikhail> why cannot we always allow web developer request frequency?

<dom> because some platforms don't support polling for some sensor types

<dom> and use cases often don't require a specific frequency for these sensors (at least, that's how tobie justified that approach)

<riju> shalamov: I explained how we do android and ChromeOS for ambient light to the group in the room

<mikhail> if platform does not support it, it should be ignored it's a hint, right?

<dom> mikhail, I suggested this as some point in the conversation; but I think Tobie's worry is that it makes the API less reliable

<dom> but mikhail, I think that's probably worth raising as a separate issue to make sure we get a clear perspective on it

<mikhail> practically, we can consider frequency for every sensor type

<mikhail> for example if the user wants to receive notification just once per hour, not more frequently

<riju> I agree with frequency to be considered for every sensor type, but like chromeos we poll at that frequncy but rport only onchange

https://github.com/w3c/sensors/issues/104

tobie: is is a tough issue
... permission story not clear, ask at construction time or when start() invoked

<shalamov> riju: Could you create issue for reporting mode and frequency? Windows, Mac, CrOS, Android and Linux (iio) do require frequency hint for *all* sensor types, regardless of reporting mode. Thanks.

resuming after lunch

https://github.com/w3c/sensors/issues/104

tobie: multiple things
... should sensorState enum have nosensor value
... my take, there's errored state, and nosensor and errored are the same
... but we should distinguish different type of errors
... already user NotAllowedError if user rejects permissions
... suggest NoReadableError for no sensor existing
... two other things
... start() and stop() return promise
... and the promise rejects with a NonReadableError

dom: confused with event and promises for signaling errors

tobie: whole lifecycle is complicated
... all is async and a lot of things can happen
... a sensor starts "idle" when calling start()
... then "activating", and to "active" or "errored"

[looking at state transitions https://cloud.githubusercontent.com/assets/975872/15710445/b6a0676a-2812-11e6-88c9-5ac916a70267.png]

tobie: some transitions missing

<riju> right now start is sync, so we need the state change, when and if we make start return promise, state changes might not be necessary

tobie: proposal, make start() return a promise
... or have an object that is observable

dom: promises are fine when the lifecycle is simple

kenneth_: what's the developer benefit?

<riju> why not add a static requestPromise which returns a promise only for one-shot reading

<mikhail_> riju: what's the problem with current setup (start + state changed) ?

<mikhail_> imo state is more verbose on what is happening to sensor..

<mikhail_> reg https://cloud.githubusercontent.com/assets/975872/15710445/b6a0676a-2812-11e6-88c9-5ac916a70267.png state can swich to 'errored' also from 'active'

<mikhail_> so, right now web developer can have a single callback for 'onstatechange' event, if promise is returned s/he would still need to have 'onstatechange' event callbacks + also watch promise results

<riju> Tobie thinks start should use promise

<riju> apparently promises play well with requestAnimationFrame()

<mikhail_> riju: still would be nice to know which promises would bring to sensor APIs

<mikhail_> *which benefits

<dom> Writing Promise-Using Specifications

<riju> Adding promises to start() and stop(): we really need to add a pros and cons before changing anything

<riju> look modern is not a good excuse to change to promises

<dom> [not to throw further confusion to this topic, but shouldn't "start" be "activate" as a result?]

anssik: proposal, drop onstatechange and enum SensorState
... web developers can build their own state machine if needed
... make SensorState an internal slot to be referenced in spec prose

resolution to #104: https://github.com/w3c/sensors/issues/104#issuecomment-248316955

Web of Things IG joint meeting

<tobie> ScribeNick: tobie

WoT

johannes: CG, trying to launch a WG
... we have slides:
... problem: is there are a lot of siloes
... basic idea is how can we bridge these.
... standards-wise we look for (low-level) building blocks
... these might be slightly domain-specific
... we're looking at common cross-cutting issues
... focus is on the application-level
... design goal is to complement not compete against existing standards
... we care against interop across different platforms
... need to support different protocols
... How can we ensure we support different protocols but still provide interop?

<maryammjd> http://w3c.github.io/wot/current-practices/wot-practices.html

johannes: we have "Things"
... We try to express things as a resources with state and limited actions

gmandyam: is the assumption underlying protocols can map easily to HTTP?

<gmandyam> Summary: WoT assumptions for protocol mappings require compatibility with HTTP semantics, resource discovery, eventing, and the ability for WoT devices to serve as both clients and servers.

<kaz> wot current practice doc

<zkis> https://github.com/w3c/web-of-things-framework/tree/master/documents

<zkis> http://w3c.github.io/wot/architecture/wot-architecture.html

<riju> gmandyam: Higher end devices with secure context is fine, but for a lot of lower end devices secure contexts is an overkill

<anssik> https://www.w3.org/wiki/TPAC2016/SessionIdeas#WebIDL_Future_Work

Johaness: what about WebIDL?

<yingying_> similiar presentation slides

anssik gives short description of what the DAS WG does re sensors.

<Tomoyuki> DAS WG http://www.w3.org/2009/dap/

<Tomoyuki> Generic Sensor API https://w3c.github.io/sensors/

<riju> a blog sharing our experiences with the generic sensor api story, https://01.org/chromium/blogs/riju/2016/generic-sensor-api-javascript-powered-platforms

anssik: workmode, we'd love your feedback, but we might not be able to take it into account.

<gmandyam> Geolocation meeting agenda: https://lists.w3.org/Archives/Public/public-geolocation/2016Sep/0006.html

Geolocation WG

anssik: what's the plan with the Geolocation WG

gmandyam: charter ends at EOY
... last to do on our list is move DeviceOrientation to rec (no API changes, just process)

<gmandyam> Remaining DeviceOrientation issues: https://github.com/w3c/deviceorientation/issues

<anssik> gmandyam: one shot request is a feature missing from the current DeviceOrientationEvent

<anssik> https://github.com/w3c/deviceorientation/issues/11

<anssik> ... also beneficial for Geolocation API

<anssik> tobie: this suggests we should add one shot request to Generic Sensor API

<anssik> https://github.com/w3c/deviceorientation/issues/4

<anssik> [tobie explains low-level and high-level sensor distiction]

<anssik> tobie: high-level sensors could be handled in hardware, e.g. via a sensor hub

<maryammjd> Anssik: discussion on the coordination system, in a phone it is according to its screen, what about a laptop, a watch, other sensor-enabled devices without a flat screen?

<anssik> currently local coordinate system is defined as https://w3c.github.io/accelerometer/#local-coordinate-system

<maryammjd> Kenneth: what is the default? portrat or landscape?

<anssik> non-normative note: "The local coordinate system of a mobile device is usually defined relative to the device’s screen when the device in its default orientation (see figure below)."

<maryammjd> @gmandyam: https://bugs.chromium.org/p/chromium/issues/detail?id=598674

<anssik> anssik: as long as the local coordinate system is fixed we should be fine

<anssik> ... the problem with DeviceOrientationEvents AFAIK is the spec was not clear on whether the coord system was fixed and implementation did it differently

<gmandyam> DeviceOrientation issue on orientation with respect to fixed frame of reference: https://github.com/w3c/deviceorientation/issues/4

<gmandyam> (Abandoned) Geofencing API: https://w3c.github.io/geofencing-api/

<anssik> anssik: Generic Sensor API exposed to Service Worker context could be used to address Geofencing API use cases

<anssik> ... or more precisely, a subset of the Generic Sensor API, namely events that deliver SensorReadings

<anssik> tobie: this could work like that

<anssik> polyfill for Geolocation API exists https://github.com/w3c/sensors/blob/gh-pages/polyfills/geolocation.js

<anssik> scribeNick: anssik

Concrete sensor specific issues

Ambient Light https://github.com/w3c/ambient-light/issues

https://github.com/w3c/ambient-light/issues/15

https://w3c.github.io/sensors/#browsing-context

"Sensor readings must only be available in the top-level browsing context to avoid the privacy risk of sharing the information defined in this specification (and specifications extending it) with contexts unfamiliar to the user. For example, a mobile device will only fire the event on the active tab, and not on the background tabs or within iframes."

Feature Policy should perhaps also consider deviceorientation https://wicg.github.io/feature-policy/#features

currently it does not

Proximity https://github.com/w3c/proximity/issues

<tobie> https://w3c.github.io/permissions/

no burning issues

Accelerometer https://github.com/w3c/accelerometer/issues

https://github.com/w3c/accelerometer/issues/3

also for this spec, the only non P&S issue is related to definition of low- and high-level sensors, whether to split into Accelerometer and LinearAccelerometer (w/o gravity)

Gyroscope https://github.com/w3c/gyroscope/issues

no F2F issues to discuss

Magnetometer https://github.com/w3c/magnetometer/issues

https://github.com/w3c/magnetometer/issues/3

anssik: this proposal seems to makes sense, should also update Gyro and Accelerometer similarly
... any objections?

[no concerns]

Adjourn

Summary of Action Items

[NEW] ACTION: Kenneth to discuss with the TAG why workers are needed [recorded in http://www.w3.org/2016/09/20-dap-minutes.html#action01]
 

Summary of Resolutions

[End of minutes]

Minutes formatted by David Booth's scribe.perl version 1.144 (CVS log)
$Date: 2016/09/30 13:30:47 $