W3C

– DRAFT –
Devices and Sensors WG - 2021 Q2 virtual meeting - Sensors

08 April 2021

Attendees

Present
Andy_Carle, Anssi_Kostiainen, Arnaud_Mandy, Kenneth_Christiansen, Lars_Knudsen, Marcos_Caceres, Raphael_Kubo_da_Costa, Reilly_Grant, Vincent_Scheib, Will_Morgan
Regrets
Fuqiao_Xue, Thomas_Steiner
Chair
Anssi, Reilly
Scribe
marcosc, reillyg, scheib

Meeting minutes

Accelerometer

Anssi: Raphael will update us on what is happening in Gravity Sensor-land.

Raphael: In terms of specifications there hasn't been much going on but from an implementation perspective the Gravity Sensor is going to be shipping in Chrome 91.

Raphael: We didn't need to make any specification changes in order to ship but we did write an explainer.

<anssik> GravitySensor explainer

Raphael: The use cases section of the explainer can be merged back into the main spec.
… WPT tests were already present.

Kenneth: It makes sense for Gravity Sensor to be part of the Accelerometer specification because it is a derived sensor from the accelerometer.

Raphael: This work is a testament to how well the Generic Sensors framework works.

Anssi: I'm happy to see a well-known customer ask for this feature.
… This was a request from Unity.
… It would be useful now to try to get reactions from other browsers.
… Have there been formal signals from other browsers?

Raphael: I don't think so.
… I think it would be useful to complete the S&P review work discussed yesterday before reaching out again.

Anssi: What's interesting about the Gravity Sensor is that this is a new surface.
… You could polyfill it using the existing API but it wouldn't be very efficient.

Kenneth: It would drain your battery.
… You can do a low-pass filter, or subtract the linear accelerometer from the accelerometer. That is a lot of processing per event.

Anssi: Are there any general thoughts?

<anssik> proposed ACTION: Arno to Merge GravitySensor explainer back to the master explainer and/or spec

<anssik> proposed ACTION: Arno to Merge GravitySensor explainer and its use cases back to the master explainer and/or spec

Action: Arno to Merge GravitySensor explainer and its use cases back to the master explainer and/or spec

<anssik> Permissions Policy Integration

Anssi: Next topic, Permissions Policy integration.

anssik: Mike Smith proposed to add Permission Policy policy specifically to accelerometer.

anssik: there is some boilerplate text for how specs do this. Because this spec is new, it needs to be done differently. I made some comments to that regard.

<anssik> The current integration is through https://w3c.github.io/sensors/#check-sensor-policy-controlled-features that is called into from https://w3c.github.io/accelerometer/#construct-an-accelerometer-object (and other sensor APIs that inherit from Sensor).

<anssik> The feature strings are defined in https://w3c.github.io/permissions/#permission-registry

<anssik> It seems if we'd make the following default allowlist normative in the Generic Sensor API (from https://w3c.github.io/sensors/#permissions-policy-api):

<anssik> The features' default allowlist is ["self"].

anssik: I proposed that if we want to make permission policy integration part of the Generic Sensor API, we would need to just add the "allow list"

anssik: as "self"

anssik: reillyg, what do you think... should we make it generic?

reillyg: I'm trying to understand Mike's request... we have the default allow list in Generic sensor, and we have the "check if allowed to use" in the Generic spec, and we have it in the other specs too. Maybe we need more non-normative text explaining how the policy integration works?

reillyg: but it seems "technically" all the right bits are there

anssik: the problem might be that it was currently non-normative. So I was proposing to make the permission policy stuff to be normative.

reillyg: it seems like a bug in the spec that the permission and the permission policy was in the extensibility section and not in the main body of the spec

reillyg: I think what we've done here is defined the permissions stuff both normatively and non-normatively, so we should make everything normative.

anssik: Yes, that would give us more control over things. But in general, we should be strict about the permission policy stuff

<anssik> Extensibility section notes: "Note: This section and its subsections provide guidance to extension specification authors using normative language. From implementers' point of view, this section and its subsections are considered non-normative."

<anssik> https://w3c.github.io/sensors/#extensibility

reillyg: the scribe made a mistake , for permission policy, it's currently non-normative - so we need to fix that to be normative.

anssik: the architecture is somewhat unique, with base classes etc... so it makes things a little bit confusing. But it's all fixable.

reillyg: the Generic sensor spec defines the "default default allow list", but other specs might be able to override that allow list.

marcosc: would it be a good a idea to allow that?

reillyg: maybe not. Maybe we should lock it down until someone needs to override it

anssik: reillyg, what is your resolution... is it to make allow list "self" for Generic Sensor API?

<anssik> proposed RESOLUTION: Make default allowlist "self" for the Generic Sensor API

<anssik> proposed RESOLUTION: Make default allowlist "self" for the Generic Sensor API, include non-normative language in concrete sensor specs to make it clear the feature is Permissions Policy controlled feature

reillyg: The problem with generic Sensor API is that it might be hard for developers to know what's going on with regards to permissions... so we could add a non-normative note in each spec to make it clear.

<anssik> proposed RESOLUTION: Make default allowlist "self" for the Generic Sensor API, include non-normative language in concrete sensor specs to make it clear those features are Permissions Policy controlled features

<reillyg> +1

+1

Resolution: Make default allowlist "self" for the Generic Sensor API, include non-normative language in concrete sensor specs to make it clear those features are Permissions Policy controlled features

anssik: next topic, Proximity

Proximity

anssik: I want to bring in feature request/clarification from an ECMA-53 group for embedded systems

<anssik> TC53 ECMAScript modules for embedded systems

<anssik> Behavior of distance attribute when sensor cannot provide distance measurements? (issue #44)

anssik: the way the spec is currently defined is incompatible with the stuff that TC53 is doing. The spec is a bit confused around the behavior of distance attribute when sensor cannot provide distance measurements.

<anssik> https://github.com/w3c/proximity/issues/44#issuecomment-803227496

anssik: Andy proposes that the behavior should return `undefined`

<anssik> The distance attribute

marcosc: it seems strange to return `undefined`

reillyg: 3 states

reillyg: No capability

reillyg: No object detected

reillyg: TC53 proposing using undefined

marcosc: undefined is concerning

marcosc: Geolocation uses:

<marcosc> Example from Geo: "Initialize coord's heading attribute in degrees, or null implementation cannot provide heading information. If the hosting device is stationary (i.e. the value of the speed attribute is 0), then initialize the heading to NaN."

reillyg: NaN used for heading when an object is not moving, it is mathematically undefined.

reillyg: many specs use 'null' for can not provide.

reillyg: need another number for 'distance undetermined'

anssik: Or, infinity

anssik: But, TC53 is concerned about these values.

reillyg: From issue comment, "desire to avoid special values that still satisfy the condition typeof value === 'number', out of concern that anything that is a number might be interpreted as a valid distance reading. This lead our discussion away from using positive infinity as a special signal value."

anssik: This is a recommendation from TC53, but we should work to be consistent with web platform APIs

reillyg: What about 'undefined', false, or number?

reillyg: Want to satisfy javascript 'truthiness', e.g. an object at a distance or just present

reillyg: For 'falsy' there are object not present or capability not present

reillyg: Wait -- if no capability, should sensor creation fail?

<Zakim> anssik, you wanted to note there should perhaps be a general best practice for this API design consideration to ensure consistency on the web platform, is there a TAG position?

anssik: Issue seems larger than just this specific API. Should have a general best practice for this situation for consistency on web APIs.

kenchris: File an issue for TAG

<kenchris> https://w3ctag.github.io/design-principles/

marcosc: Ping Domenic Denicola

<anssik> proposed ACTION: Anssi to ping Domenic to seek guidance on API design consideration for distance attribute

reillyg: One more proposal for behavior:
… null 'distance' for no capability or no object
… check 'near' attribute to distinguish

<anssik> proposed ACTION: Anssi to ping Domenic to seek guidance on API design consideration for distance attribute considering consistency with the web platform

<marcosc> https://github.com/w3c/proximity/issues/44#issuecomment-815469205

marcosc: [called for Domenic's help on issue]

larsgk: Web app developers unlikely to have devices with proximity only with no distance measurement.

larsgk: Perhaps a set of states instead of 'near' 'distance', similar to Geolocation.

anssik: e.g. 'near', 'not so near', 'far'?
… would allow a state 'we don't know', e.g. in the TC39 situation

reillyg: No implementation so far.

reillyg: The 'near' attribute not well defined, because developer can't communicate what 'near' means.
… constructor could take parameter for what is considered near
… For sensors that can only detect a boolean presence, the parameter would be ignored
… Sensors with range proximity could use the parameter as the threshold for the results.
… Might make this parameter mandatory to avoid web-app developers not planning for the various sensor types.

reillyg: Either a single threshold. Or a 'rising' and 'falling' threshold for which range the object is considered present
… only applications that need measured distance would use it

anssik: This would be a simple javascript adapter

anssik: Fundamental considerations are raised by this topic.

reillyg: Lack of interested web platform clients limits our understanding of requirements.

<anssik> https://github.com/w3c/proximity/issues/44

<anssik> near and "user presence state" interaction

anssik: Noting that recent devices have more sophisticated detection sensors.
… e.g. knowing that the object in proximity appears to be human

reillyg: [summarized previous discussion]

Andy: Was discussed, but not clear what to do with 'distance'

reillyg: Dictionaries can return undefined in web, but attributes don't typically do this

Andy: Trying to work with 'dictionaries in' 'dictionaries out'

reillyg: Another option to have return type be a union, returning number or bool, etc.

reillyg: And, highlighting larsgk concern of libraries attempting to work in both situations. Hence our possible near state being 3 enum states.

Andy: Common scenarios likely to match that proposal. E.g. applications that simply want an interrupt situation.
… e.g. configure sensor with a value and have a pin value be triggered by that.

reillyg: Current spec supports both polling and event API usecases.
… guessing that DOM / event style events not adopted in TC53

Andy: Planning to do a first pass on sensor types as generic as possible
… e.g. temp sensor only has a temp sensor and it can be sampled. But no thresholds or interrupts, or unit conversions.

Andy: Assumes the developer knows the exact sensor under the API
… opposite of web platform approach which is abstracting the sensors.
… Have a document type which allows injecting callbacks for particular sensors.
… As more manufactures get involved with ECMA we hope to abstract out of specific parts.

anssik: Need to wrap discussion for time. Let's continue discussion on issue. (others agree)
… Pleased to see TC53 engaging on this API.

anssik: Spec should be consistent with web platform standards

scheib: How soon does TC53 want this?

Andy: First version of spec is frozen and being reviewed
… v1 working to go through ECMA now
… improvement planned though mid-summer

<anssik> proposed RESOLUTION: Continue work with TC53 Committee to learn from their Proximity sensor implementation experience and refine the spec accordingly

Resolution: Continue work with TC53 Committee to learn from their Proximity sensor implementation experience and refine the spec accordingly

Ambient Light Sensor

<anssik> MVP requirements for ALS and Screen Brightness

Screen Brightness API

Will: Company desires to understand if the person using hardware is a human, not a deep fake etc.
… Using light to determine if the person is real.
… Flashing different colors.
… Predicting first if operation will be successful.
… Being a strong light it may not be, and must request user to move areas
… Or, may need to adjust intensity of light flashes.
… Ambient light helps with detection
… Screen brightness can help adjust flash levels
… Some have proposed CSS media queries
… Have also tried to read exif data from media stream.
… Some implementations don't allow reading that data without setting a value.
… Ideally would read light conditions before requesting webcam.
… Reduces privacy exposure by only asking for webcam when needed.

marcosc: Ambient sensor has had several privacy concerns with the API.
… e.g. detecting which TV show is currently displayed in area.
… The purpose of the application described as a use-case is also a significant privacy concern in that it might help identify people.
… There is a presupposition that this is the method needed to solve the use case.
… Web authentication, and secure payment APIs have similarities.
… So -- would like to life conversation to explore how we can solve the problem generally.

Will: Desire to solve this issue without using particular sensors/solutions from specific companies.
… Would like to understand the privacy concerns more.

marcosc: Firefox removed implementation due to privacy concerns.

reillyg: Noting that ambient light sensor isn't being used for identification. Only predicting success

<anssik> Status report of ALS privacy concerns and their mitigations

Will: Do not need high precision. Roughly 2 readings a second within 50 lux.

marcosc: Would want a risk assessment to understand if a malicious application could abuse the sensor in a similar way.

Will: The specification seems to be stuck, what is the status?

reillyg: Implementation paused in chromium

anssik: Some data driven investigation was not able to confirm the concerns raised by Mozilla
… w3c privacy review was unable to validate concerns, unblocking progress for previous concerns.
… current blocker is awaiting strong use cases.

reillyg: Heard high value clarification:
… Camera can't solve the problem alone. Need better calibrated light levels that ambient light sensor can provide
… All sensors currently are exposed without requesting permissions (except in Safari)
… Not ideal, would like to find a way to transition sensors to a permission system
… Hesitant to add more sensors without permission
… Camera permission an option
… In combination with other privacy mitigations (limiting accuracy)
… Suggest we circulate a proposal through wide review with the camera permission + mitigations.

Will: Had similar thoughts to augment the Media stream with an ambient level stream.

reillyg: There's a permissions spec that includes camera permissions

<anssik> Permissions spec

reillyg: Sensor specs augment the permissions spec

<anssik> Permission Registry

reillyg: Fits well to have ambient light sensor spec also reference the camera permission.
… May have other concerns, e.g. does ambient light reading warrant browser interface showing camera usage

<anssik> (Those comments without attribution were from Reilly)

reillyg: Unsure if there's any additional channels for get user media, e.g. IR cameras

Will: How will this move forward

anssik: We'll form a proposal to circulate to e.g. WebRTC

reillyg: In Chromium context, would take this to privacy review etc. Value in spec forum proposal and wide review.

<anssik> proposed RESOLUTION: Define Ambient Light Sensor Permissions API integration combining this feature with "camera" permission

Will: What will the proposal look like? Another media track along with media? Will need to contemplate offline and consider making a pull request.

Will: If not feasible to put behind getUserMedia is there a framework we could use?

reillyg: There's a desire to put sensors behind a permission. But no cross vendor idea for how to do this - issue with compatibility change.
… explaining sensors to users has stalled progress

anssik: Don't believe we've bundled permissions before

Will: Indeed that's a risk

<anssik> proposed RESOLUTION: Explore Ambient Light Sensor Permissions API integration combining this feature with "camera" permission

Resolution: Explore Ambient Light Sensor Permissions API integration combining this feature with "camera" permission

Summary of action items

  1. Arno to Merge GravitySensor explainer and its use cases back to the master explainer and/or spec

Summary of resolutions

  1. Make default allowlist "self" for the Generic Sensor API, include non-normative language in concrete sensor specs to make it clear those features are Permissions Policy controlled features
  2. Continue work with TC53 Committee to learn from their Proximity sensor implementation experience and refine the spec accordingly
  3. Explore Ambient Light Sensor Permissions API integration combining this feature with "camera" permission
Minutes manually created (not a transcript), formatted by scribe.perl version 128 (Thu Mar 4 11:59:56 2021 UTC).