Devices and Sensors WG - TPAC 2021 vF2F Day 3

29 October 2021


Anssi_Kostiainen, Arnaud_Mandy, Chris_Needham, Fuqiao_Xue, Juha_Vainio, Lars_Knudsen, Lukasz, Marcos Caceres, Maryam Mehr, Matt_Reynolds, Nick_Doty, npdoty, Peter Bruhn Andersen, plh, Raphael_Kubo_da_Costa, Reilly_Grant, sathya, tomayac, weiler
Anssi, Reilly
anssik, reillyg

Meeting minutes

Privacy, security, and architecture

<Maryammjd> Good morning (noon, evening) all!

weiler: I work at MIT for the W3C on privacy.

Lukasz: I work on research on privacy and security.

npdoty: I am at the Center for Democracy and Technology.

Maryammjd: I am a researcher at Newcastle University in the UK. I work on privacy and security issues related to sensing technology.
… Today I will be talking about a recent user study I have done on these topics.

AB/TAG experiment recommendations update

Report of AB/TAG experiment

2021 Q2 virtual meeting session

anssik: At the Q2 meeting we made a number of commitments related to security and privacy reviews.

DONE: Republish WDs of Battery Status API, Proximity Sensor, Ambient Light Sensor, System Wake Lock, Geolocation Sensor, Orientation Sensor with a note warning readers that these specifications are being reviewed with reference to the AB/TAG decision and set up automatic re-publishing.

DONE: Apply editorial modernizations to the Battery Status API, perform a round of self-review and revisions on the Security and Privacy aspects of the API before requesting horizontal review from the PING.

WIP: Investigate retrofitting the new use cases enabled by the Geolocation Sensor API to the Geolocation API

DONE: Publish a FPWD of the Geolocation API

WIP: Pending completion of other horizontal review work, revise Security and Privacy sections of the Generic Sensors specifications, provide the TAG with a revised explainer for motion sensors and seek horizontal review for already shipping Generic Sensor API, Accelerometer API, Gyroscope API and Orientation Sensor API.

anssik: I would like to focus on this last part because it is the next work ahead of us.
… plh, what are your thoughts on our progress on these commitments?

plh: Where are the horizontal review requests for these documents?

anssik: The reviews have not been requested yet. The Security and Privacy documentation still needs to be revised.

plh: I'm not going to be able to point the Council at progress on reviews because it hasn't been started.

<npdoty> is there some particular timeline in the Council or otherwise in this plan?

<plh> npdoty, we're waiting on the council to take on the objection. no timeline yet.

anssik: Question to weiler: What can we do in order to make the review process easier for PING?

<npdoty> plh, oh, I thought this December 2020 comment was the Council recommendation. are they going to provide further input again?

anssik: Is it the right approach to algorithmic privacy mitigations as well as a summary in the considerations section? Should the consideration section have its own normative requirements?

<plh> npdoty, there is a new objection on the proposed revised charter

marcosc: Putting mitigations into the algorithms is better than a generic "don't allow fingerprinting" normative requirement in the S&P considerations section.
… Would the PING appreciate both algorithmic mitigations as well as a discussion of those mitigations in the S&P considerations section?
… This does create duplicate work and the potential for specification text to get out of sync.
… In particular non-normative descriptions which don't translate to testable assertions.

weiler: I see review as a non-mechanical process. The reviewers are coaches. The best reviewers are the editors because they know the subject area.

<npdoty> yeah, I think normative requirements inline and then use priv/sec considerations to explain the threats and overview how they're mitigated (referring to those other sections)

weiler: The questionnaire is meant to guide your thinking. The TAG requires it but the PING is looking for documentation of what you've figured out while considering those questions.
… Ask for feedback as early as you can, while there is still opportunity to change things.
… I agree with marcosc that mitigations are defined algorithmicly.
… Two categories of recommendations:
… 1. Make users look the same.
… 2. Make users look different to different origins.
… (Additional recommendation for this group) Be careful of ephemeral fingerprinting.
… Two sites looking at the same sensor values can determine that the user is the same.

npdoty: I think the re-review idea from the AB recommendations is a really good review.
… Geolocation API was one of the first APIs I looked into that raised a number of permissions questions like what we are thinking about today.
… Since we are looking at a lot of sensor specifications we should use this as an opportunity to pull out the most general principles.

anssik: The Generic Sensors API defines a baseline for these sensor APIs and the baseline privacy mitigations for all of these sensors.
… We should make sure that this specification provides good mitigations so that each sensor API doesn't need to rebuild these mitigations from the ground up.
… However, each sensor should be considered for its own individual privacy considerations.

npdoty: Information disclosure is our primary privacy risk.
… This disclosure may be obvious or non-obvious.
… We should also consider carefully the ways in which these API allow a site to actuate capabilities of the device (e.g. vibration, screen brightness).
… We need a lot more work on whether users understand permissions and now long permissions should last.

Lukasz: I had the opportunity to extract some general considerations from analysis of APIs like battery and light sensor.
… I believe that these takeaways are applicable to other API concepts.
… When reading the latest Network Information API's S&P considerations I am happy to see the minimization of data collected.

<npdoty> we should definitely have on our list going through some academic work (lukasz, maryam, others?) and see if we have those general considerations/principles

Lukasz: The question I would pose is, what is the progress on the Network Information API and what major things should we be looking at?

reilly: any ideas whether the mitigations proposed by Idle Detection API are sufficient?

<npdoty> PING definitely had a lot of serious open questions about network-info when it was at the CG level, but I'm encouraged that we should expect it to be better filled out when we next look at it

<npdoty> (I also haven't looked at Idle recently; that was certainly one that had the threat of simultaneous-event firing, which weiler referred to earlier in the category of ephemeral fingerprinting)

weiler: To marcosc's earlier question: Would you like us to document mitigations in two places in the specification?
… Think of the S&P section as a table-of-contents.

<marcosc> :)

weiler: It will contain an analysis of the considerations and pointers to the mitigations.

anssik: An explainer for how to write an S&P section would be appreicated.

weiler: The IETF has good documentation in RFCs for now to do that.

<npdoty> that's great feedback. we should include the RFC pointers, or have a friendly version for how to write a good web privacy and security considerations section

Maryammjd: We discussed previous how Android and iOS were handling sensors in contrast to how the web is handling mitigations.

<weiler> [linked from https://www.w3.org/Guide/documentreview/#how_to_get_horizontal_review ]

Maryammjd: How are we trying to accommodate all these approaches and how are we considering the ways that other platforms are handling these issues.

<weiler> [ https://tools.ietf.org/html/rfc6973#section-7 https://tools.ietf.org/html/rfc3552#section-5 ]

<npdoty> I think sometimes the user experience just will be different between platforms. the Web has different affordances, and I think we often prefer how the Web does it (not assuming that apps are vetted, etc.)

reilly: challenge we're facing in browser implementations is the native platforms we're building on top of are changing

<npdoty> but I also hope we can learn from what works well in macOS, windows, android, ios -- like providing good inline explanations of purposes in permissions requests, for example

reilly: Android has done changes in terms of sensor accuracy
… rates in Android are higher than what spec requires, e.g. Android 200 Hz, spec 60 Hz

<npdoty> Maryammjd, do you have some good examples we can read from user studies on those confusions?

reilly: vague consensus build across native platforms on what the mitigation baseline should be
… we had a permission in generic sensor spec, not fully rolled out in specs, we should be able to keep the model of the web similar to what the model is on native platforms

<Maryammjd> @npdoty, yes, I present in my talk today

reilly: web platform should always be strictly safer due to the nature of how the platforms are layered

Maryam: are you in contact with Web of Things?

reilly: during Q2 F2F we had ECMA TC member talking about this topic

Behavior of distance attribute when sensor cannot provide distance measurements? #44

anssik: That was TC53.

Maryammjd: It's good to have the bigger picture of how sensors are being used in the IoT space.

larsgk: I've been working on adoption of device APIs in the enterprise space.
… I am concerned that what is being done in the consumer space doesn't align with the enterprise space.

<npdoty> oh, gosh, I thought we had been brainstorming already ;)

Privacy cross-group brainstorming

How to get horizontal review

<npdoty> yes, we definitely want folks to feel welcome to come discuss and request reviews

reilly: we seem to be on track to satisfy AB/TAG recommendations and our minutes today contain good references

<npdoty> yes, absolutely, thanks for raising that aloud Maryammjd. diversity of participation is an ongoing challenge in W3C and other Internet governance settings

[Slides will be shared after the meeting]

[Slide 1]

Maryammjd: This work is currently under review for a conference. I will discuss with my coauthors and prepare a public version.

[Slide 2]

Maryammjd: These are ambient sensors found in mobile devices.

[Slide 3]

Maryammjd: We have been looking at the risks these sensors provide alone and in combination.

[Slide 4]

Maryammjd: It is not shocking to learn that users are not familiar with these sensors and there is significant disparity between actual and perceived risk.
… Geolocation has been better studied, ambient sensors less so.
… Smart building studies have covered ambient sensors to some extent.
… At TPAC 2019 I pointed out some research into using AI/ML for managing sensors but there have been no user studies in this space.

[Slide 5]

[Slide 6]

[Slide 7]

[Slide 8]

Maryammjd: Users are not familiar with ambient sensors and are not concerned. Around 50% of participants are not concerned at all.

[Slide 9]

Maryammjd: Although they are not concerned user feel very annoyed if sensors are accessed without their permission.
… Could be install-time or at runtime. First time or each use.

anssik: Was the question framed in terms of all of the sensors from slide 2?

Maryammjd: Yes, we gave a description of the ambient sensors.

[Slide 10]

Maryammjd: Users expected native apps to have access to ambient sensors but not websites.
… Expected the OS to handle sensor access automatically in a privacy-preserving way.
… Some users preferred being notified over being asked permission.
… Others preferred an opt-in over an opt-out approach.
… Users would prefer sites to justify their use of sensor.

[Slide 11]

Maryammjd: Users were specifically worried that ambient sensors would reveal location.
… Participant identified a concern over their child's safety, recognizing that these sensors are also present on devices such as toys.

[Slide 12]

Maryammjd: Generally the actions users take to protect themselves are consistent across apps and websites.
… With the exception of closing the site being more common than closing an app, with uninstalling the app being correspondingly more frequent.

<npdoty> that's a big privacy advantage that web can have other platforms (closing the website as a mitigation) -- if we ensure that it actually does still make a difference that web sites can't get background access

Maryammjd: Users would like some notification of sensors being used.

[Slide 13]

Maryammjd: Users would like a sensor privacy management system on their device.

[Slide 14]

Maryammjd: Users want control and usability.

[Slide 15]

Maryammjd: Users are receiving too many notifications about security and privacy, many of them fake. Users would like communication through other channels about these real risks.

[Slide 16]

[Slide 17]

Maryammjd: If the notification is annoying then it is no good.

[Slide 18]

Maryammjd: Male participants expressed more knowledge about sensors and risks. Female participants expressed more concerns in relation to their privacy and security.
… No difference in level of annoyance or protective actions.
… Younger participants prefer less involvement in permission management.

[Slide 19]

Maryammjd: Given lack of permission controls in real-world systems participants feel that they just have to learn to live with the idea that they are being tracked.
… Feel that legislation should protect them.

[Slide 20]

Maryammjd: In summary, the majority of participants are not or only a little concerned about ambient sensors.
… However they would be strongly annoyed if sensors are accessed without their concent.
… Majority preferred a smart management system to handle sensors on their behalf.

<npdoty> yeah, I think we should try to learn from this and other research. I think there's a lot on permissions and how users might respond to them

Maryammjd: Note that a "take it or leave it" approach is not compliant with the GDPR.

<npdoty> yes, this is a big advantage for the Web! you can close the tab

<tomayac> A new recommendation in the Permissions spec is to show when permissions are being queried

tomayac: Permissions spec now recommends that the user be notified when permission state is being queried.
… Tricking people into granting permission has been seen in the past.

<npdoty> yeah, I raised that threat (using a permission more often once it's been granted), and I'm still not real happy about the lack of effective mitigations we have for that

<npdoty> but yeah, *ambient notice* can be helpful

tomayac: What do you think of these types of usage indicators?

Maryammjd: Inconsistency in permission models makes things difficult for users.
… Younger users are frustrated and want all the questions to go away.

larsgk: Is there a correlation between the results and users who have installed major social media applications?

Maryammjd: We didn't find a correlation between the number of apps and websites that are used and their security and privacy concerns.

marcosc: How representative was the population. I am concerned about whether these were users who did not feel that they were under threat and so would respond with "I don't care."

Maryammjd: Participants were English-speakers across EU and UK.

marcosc: We generally know that people in well-off countries won't be concerned.

Maryammjd: In human user studies we often find that what users say and do are very different.
… For example, they say they will be very annoyed or that they will protect themselves but take no action to protect themselves.
… I believe that when we design the protection mechanisms they aren't user-centered enough, contributing to these behaviors.

marcosc: Reminds me of people being surprised when people dig up very old emails when people assumed they would be deleted.

Maryammjd: We find that these kinds of wrong mental models are very common.
… We find that women feel more fear and are not confident in using these protective technologies.


<npdoty> thanks for the discussion, all

<npdoty> and thanks for the data and presentation, Maryammjd

Sensors and permission bundles, low-level vs. high-level permissions

Semantic Permission Bundles permissions #191

tomayac: this is an idea that mic and cam could go together in terms of permissions
… at some point we could define e.g. produtive app semantic bundles for things like filesystem, protocol handlers
… or maker apps for BT, USB combined
… the idea is to reduce the mental load on people who react to permissions
… clipboard is required for MS Word, you assume it is there
… lowering the cognitive load on people would help in here
… I don't not anyone on Chrome working on this, an idea at this point

reillyg: the area where this is discussed most in the WG is around whether motion sensors should be considered one high-level permission
… iOS has a single motion sensors permission
… I was in support of deprecating low-level permissions in favor of high-level permissions
… WG discussed ALS gated on camera permission as one potential way to get access to that sensor
… the general question of bundling, I'd say there's UX question in Chromium related to devices
… user's concern rel device as in a single physical device, alternative view is there can be multiple ways to access these deviecs
… how to model the device as a single entity so need to only grant permission only once
… today if you request a camera, you can access *all* cameras

marcosc: I try to delineate Permissions API from this discussion and renaming of things
… powerful features require express user permission, on the developer side, to get access to an API is a policy decision, the two come together at some point
… the mental model has been a bit unclear, but we're converging on more reasonable model now

reillyg: agree, just because access to camera does not always mean access to mic as well, this is a UX question
… asking good questions, when there are two many questions, active space of user research

larsgk: I think there's differentiation between user and enterprise
… what is MVP required for the app to work at all, optional features could come on demand

tomayac: Microsoft is considering up front permissions, in the manifest
… so that stores can display this permissions the app can ask for at some point

<tomayac> https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/main/InstallTimePermissionsPrompt/Explainer.md

reillyg: Microsoft proposal originally actually did propose up front permissions
… this opportunity to grant permissions up front if the user wanted to, "if you want the app to use cam and mic, you can grant it now of leave it for later"
… Chromium people looking at the idea of declaring these permissions up front for install time
… proposals out there trying to propose more questions at once
… there's a question in the sensor space: do we want to make this type of bundling part of the API or permissions UX?
… API modelled in terms of low-level permissions that the UA can consider to present to the user as a bundle, or high-level bundle alone -- do we want to preserve the expressiveness?

anssik: low-level gives more freedom to implementers

marcosc: I don't have a good solution to this, we can go either way
… the freedom to implementers, I agree, but don't want that to be in the spec
… in Screen Wake Lock API, for example, the permission was implemented sometimes but not always
… implementers should have freedom to experiment

reillyg: Wake Lock example: we assumed it is better to have perms spec'd
… if it'd not be in the spec, the API evolutions would be harder
… maybe too much future-proofing?
… if we think low-level is better but put high-level in the spec, is that hard to change?
… or, as long as promise-based, we can evolve without breaking existing websites that rely on current API behavior?

marcosc: these are just strings

reillyg: we should probably keep the existing permission policies as is

marcosc: developers want policy controlled features they can disable certain features
… 1:1 mapping between the strings not a requirement

reillyg: "motion sensors" permission makes sense to me, defining a singular permission is worth prototyping

<tomayac> +1 for merging the motion sensors permissions

marcosc: mental model, request a permission for each, not optimal situation

proposed RESOLUTION: Explore update to the specs to add high-level "motion-sensors" permission to encourage prototyping

proposed RESOLUTION: Explore update to the sensors specs to add high-level "motion-sensors" permission to encourage prototyping

<tomayac> +1

RESOLUTION: Explore update to the sensors specs to add high-level "motion-sensors" permission to encourage prototyping

larsgk: any work on required and optional permissions in manifest to reflect that into UI/UX?

marcosc: these enterprise scenarios would have a separate permissions policy in the browser

reillyg: from Chromium perspective, we have a policy mechanism for enterprise admins to deploy a policy to users that grants access to selected features

larsgk: works for enterprises that have Chrome for Enterprise?

reillyg: there's a cloud-based service that uses Google's management console, but policies are configurated through Windows AD, Mac corresponding client, and in some similar way for Android

xfq: For normal users, there is currently no such mechanism, right?

reillyg: you can set policies on Linux, but issues related in non-managed scenarios
… Chrome does not allow frequently abused policies to be set on non-managed devices to mitigate abuses

Ambient Light Sensor, consider making granularity of the data normative

Consider making granularity of the data normative #63

reillyg: since last discussion, we spec'd a normative privacy mitigation by limiting motion sensors to limit granularity
… ALS mitigation was not made normative, but the mitigation has been implemented in Chromium

marcosc: makes sense when you reach a baseline and adapt it as we go
… similar situation with Gamepad API, and anything to do with timing, domtimestamp etc.

rakuco: adding this to the spec is pretty easy, I was not involved with ALS when we reached conclusion 50 lux is a reasonable threshold
… need to document why we landed on this

<tomayac> +1 to (1) then

RESOLUTION: Make Ambient Light Sensor granularity of the data normative

Ambient Light Sensor, revisit developer feedback

Request for Ambient Light Sensor web developer feedback

anssik: getUserMedia misused to get data that could be provided through ALS adhering to data minimization principles

larsgk: we have a concrete ALS use case for dashboards in vessels
… they could not say "yes" to camera access but would grant access to ALS

marcosc: I wonder how much overlap there is with OS level control, CSS dark/light mode?

larsgk: we have tri-state, not boolean dark/light

marcosc: documenting a use case would help
… it feels like we have light mode, and how that translates to CSS level, are we pitching it on the right forum?

reillyg: there's an opportunity for the web to provide access to low-level capability OS does not understand
… a long-tail use cases on the system OS does not provide an API for, there's value in innovating in this space where OS vendor have not standardized APIs
… Lars' case is true, there's a context that's different from a regular user case, in Chrome we call this Kiosk device
… a sensor on the device, the permission and access model is a little different from the general user
… there's a higher level concept we want to understand, websites want to know on which light level they want to be "dusk" and "dawn"
… no conclusion other than, in Chromium we've considered a number of APIs that fall in the Kiosk mode
… figuring a best permission model is important, restricting generalization of the general consumer case

reillyg: we'd be willing to enable the ALS API for websites that use camera already
… we could consider ALS augmenting the camera

rakuco: with my Intel hat on the biggest difficulty is to know if there's enough demand for this feature
… we can land optimizations to the implement, we want to know if it makes sense to spend time on this

reillyg: we know there's demand, but how loud that demand is
… we have a few developers who'd be happy if we'd get this past the finish line

rakuco: right, for implementation specific discussions, let's follow up on that

reillyg: also spec PR to bundle ALS and camera permissions together

proposed RESOLUTION: Add camera permission requirement to ALS spec to help enable shipment of the API

<rakuco> +1

<tomayac> +1

RESOLUTION: Add camera permission requirement to ALS spec to help enable shipment of the API

Magnetometer, discuss any spec blockers for shipping

reillyg: have we gotten developer interest?

rakuco: I'm not aware

Use Cases and Requirements

reillyg: input methods involving magnets, gesture recognition, detection of direction
… not aware of other use cases?

anssik: do we have data from Android app usage of Magnetometer?

reillyg: there's value in having these specs around, e.g. Unity asked for Gravity Sensor and it was shipped quickly

larsgk: I think there are opportunities in the accessibility space, this API is possible an enabler of innovation in that area
… this API could be used to detect e.g. if you're inside a Faraday cage and will face connectivity issues

tomayac: in the context of cardboard, how did this API come to be?

reillyg: crbug from 2015 referring to cardboard device that is enabled by this API
… that issue also mentions generic sensor API, that came after

tomayac: pressing a button on a handheld device, is that the latest approach in cardboard-style usage (instead of magnet-based input)?

reillyg: there's detection of hand movement, e.g. in Meta's announcement hand movement can be used to map hand motions into virtual space

anssik: sounds like we need to modernize use cases for Magnetometer
… OT expectations, do you need developer pull to start OT?

reillyg: strongly preferred to have enthusiastic developers using a feature behind a flag first

reillyg: worth adding a note to the spec that this spec is requesting developer feedback and use cases

marcosc: metal detector use case, useful on a beach!!

<marcosc> mc: channeling PLH, we've had this published for a while. How much more interest can we expect to get for the API?

reillyg: I'd like to have an explicit status that points developers to a place to provide their feedback

marcosc: WICG is usually a place where we gather this type of feedback

reillyg: in this case we have a good design, but have "a solution looking for a problem", not incubation really

RESOLUTION: Update Status of the Document to indicate Magnetometer is looking for developer feedback and high value use cases

anssik: thank you everyone for amazing 3 days of exciting device-centric discussions!

Summary of resolutions

  1. Explore update to the sensors specs to add high-level "motion-sensors" permission to encourage prototyping
  2. Make Ambient Light Sensor granularity of the data normative
  3. Add camera permission requirement to ALS spec to help enable shipment of the API
  4. Update Status of the Document to indicate Magnetometer is looking for developer feedback and high value use cases
Minutes manually created (not a transcript), formatted by scribe.perl version 158 (Sun Oct 17 00:40:18 2021 UTC).