Meeting minutes
<marcosc> we are in https://mit.zoom.com.cn/
<marcosc> :)
Laura_Morinigo: Samsung SR UK
… Here to discuss mobile devices and foldables.
Introductions
Sangwhan_Moon: TAG Invited Expert (@cynthia on GitHub)
Agenda Bashing
<anssik> https://github.com/w3c/devicesensors-wg/issues/31
https://github.com/w3c/devicesensors-wg/issues/31
reillyg: Shuffle privacy discussions first and then follow with the technical topics which fell off of the agenda from yesterday.
AK: Move WebHID first since it has the most concrete discussion to be had.
Lukasz: I have a specific privacy concern to discuss as well.
… The Network Information API seems like a type of sensor which has privacy concerns beyond what is discussed in the Generic Sensors API.
<Zakim> tomayac, you wanted to ask about ALS
<sangwhan> ALS = Ambient Light Sensor
TS: I would like to discuss Ambient Light Sensor. Where are we today?
larsgk: Can we discuss non-goals for addressing privacy concerns as these APIs often collect a lot of FUD arguments.
AK: Historically this group started with a much broader and non-browser view of these APIs which didn't focus on privacy and that has given us a bad reputation.
MC: 2012 was a different time. We didn't have the permissions models we do today.
AK: It's not easy to fight the fake news. Let's stay professional and stick to the data.
reillyg: I think we need to apply common patterns to avoid re-litigating discussions for each API.
AK: We should liaise with the PING and we thank Lukazs for being here today.
Privacy Discussion: WebHID
MR: WebHID is the API for accessing human interface devices.
… It is shaped very similarly to WebUSB.
<anssik> WebHID API spec
MR: human interface devices are typically USB or Bluetooth but are excluded from WebUSB and Web Bluetooth.
… Keyboards and mice are excluded from WebHID because they are already covered by existing APIs and allowing low-level access would allow for keyloggers and such.
… Sangwhan raised concerns during the TAG review over the read/write capabilities of the API and the shared access model.
… The HID protocol allows devices to declare the data they provide in a standardized format. This provides high entropy.
… Devices may also provide access to data (i.e. serial numbers) which would provide high entropy.
… The API does not directly provide access to this data.
… Another concern, the API could be used to change the behavior of the device to cause malicious behavior towards the user or the host the device is connected to.
MS: Any device which allows writing can be used as a side channel and that breaks the same origin model.
… No ideas for how to mitigate this. Read-only or exclusive access only get you so far.
MR: Existing strategy is providing a blocklist to deny access to known vulnerable devices and to block access to data that looks like keyboard or mouse input.
… We are building an intake for vendors to propose new blocking rules for their devices.
… WebUSB and Web Bluetooth have similar blocklists. HID may be more granular than USB because of the report info.
Lukasz: Why a blocklist instead of an allowlist?
MR: There is a long tail of devices. An allowlist couldn't cover enough devices to make the API broadly useful.
MS: Usages to block is being communicated directly to the browser vendors?
MR: Yes.
Lukasz: A blocklist could create a bad impression.
VS: Regarding allow- vs. block-lists. We explored this for WebUSB. That API originally chose an allow-list approach and the community saw many downsides and not many upsides.
<tomayac> I think that's the Medium article @scheib was referring to: https://medium.com/dev-channel/the-webusb-security-model-f48ee04de0ab.
VS: Regarding same-origin policy, APIs such as file access already "violate" the policy and users understand that this violates the model by allowing data to be transmitted between sites if chosen by the user.
scheib: It's hard to take over a device, there's a picker model, and one that would have to be overcome for multiple origins.
… We're solving a computing need on the web, instead of on the native platform, which would be more dangerous.
<Zakim> sangwhan, you wanted to ask about how to make the deny-allow list consistently applied across implementations
sangwhan: Would like to know if there's a path forward to make the blocklists and blocked messages shareable across implementations?
reillyg: The block list is part of the repo
… It's the plan to share this
<Zakim> lukasz, you wanted to ask about user interface
sangwhan: Sounds reasonable
lukasz: I see that the spec is very closely related to WebUSB
… I wonder how the process of connecting to the device look like?
… A picker?
… For hubs(?) you would need to reassure a better reception
… Does the spec define the UI
… improve the reputation
mattreynolds: The current implementation uses the same picker model
… We can probably take some lessons from WebUSB and apply them
reillyg: The SOP aspect of this and the existence of the picker model brings this out of the SOP, it's the user bringing the device to the page and making it interact with it.
… in terms of bringing in WebUSB lessons, the article that I wrote, I spec'ed browsers should announce to the browser that there is a device that accompanies it
… so sort of bringing the SOP to this
… even for device vendors interested in this, they were reluctant, since updating devices in the field is hard
… so it was hard to make progress
… therefore we came up with vendors registering in the allowlist
… we are reasonable comfortable to accept blocklists
… like this device gives everyone access to your bitcoin
… we need to validate that the site owns a device
… the third aspect was (feedback from Mozilla or marcosc)
… it runs counter to open development and innovation
… once the vendor goes out of business, the device is lsot
… so we moved to another model.
… (see my article)
… Web Bluetooth didn't have this at the time
… so I felt bad about it for a while for Web Bluetooth being less secure
… but I was wrong
… The change was difficult to make.
… since we opened ourselves to more vectors of attack
… Can the user make security decisions with a picker
… that's why we still have the blocklist and a way to inform other vendors of this list
… we also have security key tokens
… but that attack was never used in the wild, but a security researcher found it and reported it
… but there are, apart from the security keys, no other devices on the block list.
JamesH: I like what reillyg said
… There should be something in the spec saying that there should be a picker
… sounds like lukasz would be a good person to do this
… I feel like that this is the case, but I don't have data
<Zakim> lukasz, you wanted to ask about user awareness and intentionally malicious devices
anssik: lukasz, do you have research on pickers?
lukasz: Thanks for the explanations, reillyg.
… sorry for disrupting this a bit. We all have pickers and SOP.
… It's unclear if users understand SOP.
… It should be designed well from the ground up.
… Did you consider, as another attack scenario, malicious devices being distributed to users, like thousands, that advertise a software that makes browsing bank sites more secure
… when users connect such a device, unlikely, but let's assume.
… Can we assume users will understand the risk.
… So maybe allowlists would be better suited.
reillyg: The attacks that we consider are attacks by a site on a device causes it to be malicious on its own
… for example, a site takes a device and pokes the user in the eye
… if the user can be convinced to connect a device
… that device already can do a looot of things without the browser
… The distribution of such a device doesn't change the attack vector from a UA point of view
lukasz: HID and Web USB would not change anything?
reillyg: Correct
lukasz: Would not make anything easier?
reillyg: No
anssik: Wanted to come back on picker UI research?
anssik: It's the same model as used by file pickers
marcosc: It's context driven for files
… So pickers are closely connected mentally
… unlike with device pickers
… it becomes ridiculous to ask for research on the research
anssik: otherwise it becomes opinion
marcosc: W eknow the attacks take place. If something hasn't been hacked today means it'll be hacked tomorrow
… plugging in things that were never designed for the web is dangerous
… like raw sockets, if you connect a random printer it could do bad things
… we've seen those attacks
… concrete example: the US attacking Iranian centrifuges over USB
… These cases exist
larsgk: Huge Web USB and WebHID fan
… WebHID can connect to Corsair keyboards
<scheib> Related article from a Chrome security member discussing permissions https://emilymstark.com/2020/07/14/debunking-the-users-always-click-yes-myth.html
larsgk: I can store colors, right. There's a chance someone may not understand this.
… we can consider if it makes sense to use a token based model, even if it means extra work
anssi: Let's flush the queue
<Zakim> sangwhan, you wanted to ask if exclusive access can be mandated by the spec
sangwhan Question plus suggestion: the current spec allows multiple origins to access. Is there a particular use case when you designed it this way? Could it be restricted to a single tab?
mattreynolds Would be possible, but might make the API harder
… The use cases for multiple tabs were: you might want to draw a picture of gamepad interaction in different tabs. Streamers use this to show gamepad state while playing
… Or lights used as a notification from multiple sources
<Zakim> JamesH_, you wanted to react to sangwhan
JamesH: Espruino devices have a serial port. I connect to it from different tabs. One for flashing, one for interacting with the flashed code. Doesn't work for Web USB
sangwhan This are same origins?
mattreynolds Not necessarily. Could be stadia.com and gamepadviewer.com
anssik: We're getting close to the break. Resolutiuon?
reillyg: The concerns I'm hearing: general question of malicious devices
… and I hear a concern about users understanding the impact of connecting devices, with SOP, but also general trust
… we covered allow and block lists
… We discussed this and answered questions
anssik: User studies would be much welcome
… Does Google have resources?
reillyg: We've studied permissions in general
… our experience with these APIs has been limited enough.
… we're seeing developers build interesting things. But not enough to require a deep study
anssik We have some university members in the group who are interested
… Maryam Mehrnezhad (?) and (?Ehsan Toreini)
JamesH: We have a note about using the picker
reillyg: The spec provides some language to use about the question to ask
anssik: Can you formulate a resolution?
reillyg Want to make sure this is productive for sangwhan and help the TAG.
sangwhan: I have to ask other questions through GitHub. Thanks!
scheib: There's enough appreciation for raising concerns. But also important to understand what it takes for a user to get a task done.
… Our goal is to make the interaction easy. We know users happily install high priviilege software.
… Web Bluetooth and WebHID lower this risk, but are still somewaht risky.
… Definitely more secure than installing native programs
… of larger range of permission and of longer time. Need to find a balance.
anssik: Let's break with this.
[break]
Privacy Discussion: Ambient Light Sensor
reillyg: the ALS API looks for more interest to move forward
… in Chromium we implemented mitigation to too much data exposed limiting the brightness interval to 50 lux, so can defer whether the user is in a dark room or a light room
… use cases include identifying is the conditions are good for taking a photo etc.
<tomayac> Another use case is the maps one: you drive through a tunnel during the day, and the map would switch to dark mode temporarily
reillyg: in the context of Orientation Sensor we discussed data precision and fingerprinting last year
… Chromium implements mitigations for that spec, bugs filed for Firefox and not sure what is the Safari's status
… reason for not shipping yet is not enough people asking for it to make a decision to ship -- the current state is good, we could ship it as is
… use cases for ALS with camera may need more precise readings, maybe we can provide more precision if camera permission granted alongside
<marcosc> maybe: https://bugzilla.mozilla.org/show_bug.cgi?id=1057185 bugzilla bug?
anssik: what could be done to help this API ship?
<lukasz> I wrote a small assessment of the ambient light sensors privacy design https://blog.lukaszolejnik.com/shedding-light-on-designing-web-features-with-privacy-risks-impact-assessments-case-study/ including with a paper (linked inside)
<lukasz> (if anyone finds this of interest)
reillyg: the API is available behind a flag and when developers have increased interest in it, we look into shipping
marcosc: reillyg mentioned use cases, given the current state of the world and how the cameras are used in the world
<tomayac> I don't see an origin trial for ALS.
marcosc: on the iPhone you can adjust the brightness when taking the picture?
reillyg: clarifying the use case, developer to get absolute signal for exposure so instructions can be provided to the user "please turn on the light" or "go outside"
… there's another use case to do facial recognition and use ALS to assist the user to fix lightning conditions
tomayac: another use case, drive through tunnel and it becomes dark, when you leave the tunnel switch back
… this should a reasonable use case
… discussion on prefers color scheme etc. but this is a temporary situation
reillyg: ambient light level signal could be exposed through CSS to complement the API
<xfq> https://github.com/w3c/csswg-drafts/issues/5359
Anssi: there are use cases for having both JS API and CSS feature for ALS
tomayac: there are some customer requests for ALS
Raphael: iProov company is very interested in this API
reillyg: as with the dark more, there's a bit of developer trendiness issue, if we make light level reactive sites trending more sites might take advantage of this
Lars: we have a very concrete need for this with shipping, avoiding the people to get blinded using ALS
<marcosc> We ack Mandy Michael's demo https://www.youtube.com/watch?v=ivz1hdAhJmE&ab_channel=MandyMichael
reillyg: there's a question how reactive this API should be, there's signal from the environment (what's the lighting like) and from the user (use dark mode)
<Zakim> marcosc, you wanted to say, darkmode is OS controlled
marcosc: dark more being both OS and user controlled is important
… on macOS it switched dark mode based on sunset
… it is about the balance
<tomayac> Two nice demos: https://codepen.io/arnofiva/pen/VJjxRZ and https://variablefonts.dev/posts/light-sensor-demo/
ALS Security and Privacy consideration
<marcosc> relevant gecko bugs https://bugzilla.mozilla.org/show_bug.cgi?id=1359076 - AmbientSensor was removed from Gecko
sangwhan: curious if the 50 lux stepping should be tracked in a GH issue to make it normative
<Zakim> marcosc, you wanted to say, should we consider implementer interest?
reillyg: we set developer interest first, implementer interest second
<Zakim> lukasz, you wanted to say about normative aspects of 50 lux recommendation
<marcosc> MC: agree with everything reillyg said
Lukasz: perhaps 50 lux cap should be specified in a way it is normative, but just want to point out it might be a rare example security and privacy consideration become normative?
Lars: traditional enterprises migrate tools to the web, it is dangerous if browser vendors only consider vocal developers who are well known, new domains do not have as strong voices in the process of making standards
reillyg: with Thomas we talk to many enterprise developers so we're aware of this, we agree with you that enterprises do work more behind closed doors
<reillyg> +1
<tomayac> +1
Resolution: Continue monitoring ALS API developer interest and work with more browser vendors. Encourage developers to experiment with existing prototypes.
Privacy Discussion: Network Information API
DAS WG and Web Perf WG collaboration opportunity
reillyg: when you raised this topic you mentioned couple of topics
… all the fields in the current proposed spec, as resolved yesterday, we are interested in collaboration with Web Perf WG on aspects that have to do with the signal about saving data or metered connection info
… other signals may provide more info than the site needs or poorly specified
… looking at the current users of the API, we ask, what those more problematic fields in the API are and could they be deprecated?
… moving forward we'd focus on this privacy-preserving subset and better specify other parts of the API
<xfq> The Web & Networks Interest Group is also exploring this topic: https://www.w3.org/wiki/Network_Quality_Monitoring_and_Prediction
Lukasz: ack lukasz
Lukasz: the actual API differs from the other specifications in this group, it is unclear if the Generic Sensors considerations could be casted on the Netinfo API
Generic Sensor API S&P Considerations
Lukasz: when the user is roaming, you expect these changes to happen often, an event might fire quite often(?)
… could allow profiling the user
… re RTT, I don't fully understand the semantics, is it RTT of the base station?
<Zakim> lukasz, you wanted to ask about aptness of generic sensors considerations as applied to NIA
reillyg: RTT might not survive the refactor of the spec
… marcos feedback was that Firefox has difficulty in implementing EffectiveConnectionType
<marcosc> MC: nit, EffectiveConnectionType is hard
reillyg: we're reduce the spec to "save data" and "on metered connection" signals
Lukasz: is there a way to get rid of the RTT
Lukasz: maybe too premature to assess privacy of this API at this point?
reillyg: privacy experts should be engaged when we possibly refactor this spec
marcos: without interest, it likely won't be edited
… these issues are in the Netinfo API tracker on GH, I'm not expecting any work to happen unless DAS WG wants to take this work on
sangwhan: curious how this ties into the network goodness hints that WebRTC has, and also hints one can pull out of adaptive streaming (e.g. DASH) - feels like there are three places where this information can be retrieved, all probably in different formats
reillyg: that level of information is in WebRTC and should stay there and we don't want to duplicate that in the context of the Netinfo API
sangwhan: I'm interesting in non-WebRTC use cases then?
reillyg: use case is for the user to express to the site the user is on metered connection
… this is about expressing a preference for low data usage
… the use cases have not been updated as the scope of the API shifted
<sangwhan> So summarized - the "I'm not on wifi" API
reillyg: the first order biz would be to update the use cases to match the revised scope
reillyg: the proposed refactor of the Netinfo API is to provide a signal "I'm on WiFi that is free and unlimited"
<tomayac> +1
<sangwhan> +1
<tomayac> +1
<lukasz> +1
<lukasz> (I've already provided some preliminary points)
Resolution: Engage with privacy experts in the proposed refactor of the Network Information API, figure out coordination between DAS, WebPerf, and WICG; and possible standards track adoption at a later stage
Test automation
reillyg: There was a question raised on specifying test automation
… one was to use webdriver
… the other was take the approach of Web XR to make a full Web API available for testing purposes.
… I have opinions.
… There are two audiences: 1: us as web platform community. This audience is interested in testing if the impl is correct.
… 2: web developers, who are interested if their sites work correctly.
… sometimes these interests align, sometimes not.
… If there is a sensor that is expressing a certain value, implementors want to make sure it goes through correctly.
… whereas developers just want to know if their site behaves correctly.
… The Web USB testing API is capable of emulating certain device conditions and signals.
… If you're a developer writing a site, you might not need much of this infra.
… You can just mock out the entire browser APIs.
… You can test if your site behaves properly, maybe even outside of the browser through unit tests.
… Browser automation is different.
… You probably want to loop in real devices, and detect the presence of devices.
… Without any kind of permission UI so it can be scripted.
… Implementors care about pretending devices.
… When we define webdriver extensions, it's the thing developers use when they need a browser involved.
… Those are the kinds of things I'd need to consider.
… Defining a test-only API targeted to web platform tests, and another targeted to developers: both might make sense.
… This would be orthogonal to the existing testing API.
… To a certain degree none of this matters, the goal is to provide an automation API, if it's shaped like web driver or something else, no matter
… People can use it either way
marcosc: Just wanted to say that's great. But need to balance the resources of the WG.
… If it doesn't serve the purpose of interop, I'm not sure it should be focus of the group.
… Sending the permissions API through webdriver might make sense, but, striking that balance is additional work.
reillyg: Maybe I misunderstand you.
… Disagree. Developer infra is important for interop
… If we have a way for developers to test at all, it should be interoperable.
… There is a W3C group that owns that
… It's good for developers to have them.
marcosc: My question is if _we_ design those
… Or working with another community.
… Like "click this button", etc. (describes the flow)
reillyg: If you want it done, you gotta do it yourself
… I got a lot of stars on the bugs for adding it
The question is: should it be a default, so all specs need to come with one. Or write it later when developers ask for it?
marcosc: Agree. Then the question becomes to which APIs we add this.
… If there's a strong case coming from developers if it would help, we should add it.
reillyg: Ironically currently there is just an ad-hoc testing impl.
… The interesting thing is that we have refactored the entire architecture of the device orientation API.
… This uses the same infra as generic sensors.
… But currently there are no WPT tests.
… This raises the question how to, in an ionteroperable way, to express this.
scheib: I have a humble opinion when to do the impl work for the webdriver tests
… It's faster to start early.
… Sometimes there's churn in what we need.
… There is an upfront cost.
reillyg: It's easier to say there's this API, rather than having certain endpoints APIs need to specify.
scheib: It priotizes to have implementeations of tests for multiple purposes.
reillyg: It took us forever to get testdriver.click. Why was that? It's ridiculous.
… Someone maybe back in the 90ies should have thought about this.
marcosc: To reillyg's points> scheib said including tests as we specify tests
… is hard
reillyg: Do we want to wrap up with a resolution?
<marcosc> MC: add tests as you specify
reillyg: Like webdriver not formally required
Raphael: reillyg's summary is a fair summary of the status quo
… there is no conclusion either
… there is interest in reaching a common ground.
… It's hard to commit to a stable webdriver impl if your spec is in early stage
… this should work with Web Bluetooth and -USB
reillyg: I complained last year in the testing WG that it's hard
reillyg: It might make sense to start with something less formal, that then converges into something more formal.
… Having webdriver hooks is a sign of a stable impl.
Raphael: Currently there is no process for moving to a formal model
reillyg: Yeah. There has been an effort, so these tests should work outside of the Chromium CI now.
… At least documenting the APIs should be given. Just uploading the tests without docs wasn't good.
… WebXR may have done a similar thing.
… Maybe we want a policy: to have a certain amount of documentation that explains how it works and what the expectations are
anssik: I let reillyg converge on a resolution
reillyg: Starting with my initial idea: testing should be given. We have it in our charter
… And we need to define a process
… Webdriver tests. And what counts as a truly interoperable way?
Raphael: Is the goal to make spec writer do this?
marcosc: That's a good point. Wondering if we have the expertise to do this.
Raphael: At least as we did for generic sensors.
marcosc: Who did Generic Sensors?
Raphael: A colleague at Intel
anssik: The specs are now back in WD stage
marcosc: If we have the know-how that is great
reillyg: This was my exact complaint. In my attempt, I searched around, and it was hard.
<anssik> Generic Sensor API - Web Driver extensions
reillyg: Maybe there is some more infra work that needs to be done.
… Specifying a webdriver extension isn't that hard.
<anssik> Ambient Light Sensor - Web Driver extensions
marcosc: If we have the resources to do this ourselves, that's great
anssik: Dropped a number of links
marcosc: The spec part is easy, the impl is harder in C++ etc.,
anssik: What reillyg said isn't documented
Raphael: Have reached some experience with that. It's complicated, and in addition, one would also need to implement the driver and spec it with the WPT community. It's a lot of work.
reillyg: This was my complaint. Was then told that it's not that hard by someone from Chromium.
Raphael: Talking about the WPT meeting?
reillyg: Talking about the Testing and Tools meeting
reillyg: Talking about the proposed resolution
<marcosc> +1
+1
Resolution: Developing tests in parallel to specifications is best practice. The WG should develop a policy on when to consider specifying Web Driver extensions and a policy for the level of formal specification of test-only APIs before they can be used in Web Platform Tests.
<xfq> +1
Adjourn
<marcosc> Eric, to join the group http://www.w3.org/2004/01/pp-impl/43696/join
Anssi: Thank you all for participating! Thank you scribes, speakers, we'll follow up with possible another F2F in not so distant future.
<marcosc> 🎉🎉🎉
<marcosc> ah, thanks for reminding me