W3C

– DRAFT –
Devices and Sensors WG - TPAC 2021 vF2F Day 2

28 October 2021

Attendees

Present
Alexis_Menard, Anssi_Kostiainen, Arnaud_Mandy, Diego_Gonzalez, Francois_Beaufort, Fuqiao_Xue, Juha_Vainio, Kenneth_Christiansen, Lars_Knudsen, Laura_Morinigo, Marcos_Caceres, Matt_Reynolds, Raphael_Kubo_da_Costa, Reilly_Grant, sudeep, Thomas_Steiner, Vincent_Scheib
Regrets
-
Chair
Anssi, Reilly
Scribe
Anssi, anssik, marcosc, reillyg, tomayac, xfq

Meeting minutes

Device Posture API

Revised posture types

diekus: Recall, this API allows the developer to detect the physical posture that the device is in.
… Originally presented as the Fold Angle API, providing the angle between the two halves of a foldable devices.
… Now, as the Device Posture API, supporting devices with any type of posture, regardless of whether they use a folding screen.
… Abstract postures into three groups.
… "Continuous" represents a device in a flat (or flat-ish) position.
… "Folded" represents devices that can physically fold.
… Want to differentiate between a device which is folded and one which is folded over (the third posture) which occurs when the device is folded beyond the continuous mode.

https://w3c.github.io/device-posture/#posture-types

Alexis: Simplification driven by TAG feedback.

anssik: Has the TAG approved these final posture types?

Alexis: Yes, the review was left open until the new postures were defined.
… Working on backends for more platforms. Android requires additional SDK changes.
… Windows has no OS APIs for posture (e.g. hinge angle sensor). Intel is engaged with Microsoft to determine a path.

Laura: Concerns with privacy were resolved by the latest changes.
… Current postures are working okay for Samsung.
… Some developers are still interested in the angle.

anssik: We expect that the editors will continue to evolve the specification to track the ecosystem of new foldable devices.

marcosc: Have the posture names been approved by the CSS working group?
… In particular "folded-over" includes a dash which might have special meaning.

Alexis: We have not. We can reach out to the CSS WG.

Kenneth: It was looked at by members of the CSS WG as part of the TAG review.

Alexis: I will take that action.

DAS WG Coordination with CSS WG

Alexis: Would like to keep everything in the Device Posture specification because it hasn't gotten enough developer feedback.
… Regarding the fold angle, it was removed as a privacy matter.
… The balance of use cases vs. the privacy risk wasn't compelling.
… If there are use cases from developers for the hinge angle let's open an issue to track those.
… We know that in the future the posture won't be only dependent on the angle.

anssik: It might make sense to separate these two.

Alexis: Let's provide that issue as a landing zone for developers who are looking for this.

Laura_M: One of the first things developers ask for when we explain the API is how to get the exact angle. After we explain the postures developers stop asking about the angle. We haven't found specific use cases for the angle.
… This ends up being an area of confusion rather than a developer need.

<Zakim> tomayac, you wanted to ask about more than one fold angle.

tomayac: So far this assumes that there is only one fold axis. What about future devices which fold out from both sides?

https://www.ubergizmo.com/2021/02/expanscape-aurora-7-laptop-seven-displays/

tomayac: In this example there are 7 displays. Maybe this could be handled by Multiscreen Window Placement.

Alexis: This is why we moved away from the exact fold angle.
… No matter the number of screens there is only a single posture.
… Multiscreen Window Placement can be used to understand the layout of the displays.
… Another posture could be added if a new device introduced a truly unique posture.

reillyg: do you consider this type of laptop to be presented flat in from of the user?

alexis: correct, then you'd use Multi-Screen Window Placement API

anssik: This is an opportunity to coordinate with Multi-Screen Windows Placement API.

TAG review status

Alexis: TAG review drove simplification of posture types.
… Review closed as satisfied.
… TAG suggested looking out for multi-hinged devices, which are not yet a reality.
… It's hard to know yet what they would look like and what you could do with them.

Action: alexis to add a note for possible future work to support new form factors

Semantic postures

diekus: Can this API include devices which physically change their posture (e.g. the Surface Laptop Studio)?
… but don't actually involve a screen fold.

diekus: Windows does change UI presentation based on these posture changes.

Alexis: When you switch to stage mode is the only difference from "continuous" that you cannot access the keyboard?

Device Posture API to include "Semantic Postures"? #94

Alexis: I assume that Windows triggers tablet mode and the developer can detect that using CSS media query for pointer accuracy to create a touch-friendly interface.
… Today this can be detected. Is a new posture needed.

diekus: I wasn't aware that tablet mode was detectable.

Alexis: CSS pointer fine vs. coarse.

diekus: If this is detectable then there's no need to make this a posture unless there is enough of a pattern here that this would be easier for developers.

Alexis: Developers are already used to doing this.
… We should try it with this device to make sure it works as expected.

diekus: I will go back to the Windows team with this.

<darktears> https://patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover/results/

<darktears> https://patrickhlauke.github.io/touch/pointer-hover-any-pointer-any-hover/results/

reillyg: the same comment as alexis, I think this goes back to an earlier question, what the use cases were for detection each of these posture modes?
… want to understand how folded over differentiates?

alexis: one use case is a game with two sides, for example

kenneth: multiplayer game

reillyg: in Studio device, in slate mode, is there a way to detect that stylus mode is active?

alexis: when the device is flat and using pen as the primary input, CSS properties will return pointer-coarse

reillyg: questions for diego: is there a developer need to differentiate between stage and studio modes?
… is adaption the UI something the developers are looking for?

diekus: I'll go back to Edge team to ask

alexis: user interaction issue, you assume the pen is primary, but the computer does not know that unless the pen is close to the computer
… e.g. could be out of battery
… there's a problem with UX changing from touch to pen and back

reillyg: similar situation in stage mode, when keyboard is covered but trackpad isn't

reillyg: need to go ask developers whether this is something they want to differentiate

Laura_M: I will add an issue so we can collect developer feedback on this.

<Laura_M> https://www.youtube.com/watch?v=f9Bwc_Hqfcc

Action: diego to open a GH issue to collect developer feedback on whether they want to differentiate pen input from touch input

Screen Wake Lock API

Wide review tracker

anssik: Marcos and Raphael have been doing good work here.
… All reviews have been requested and resolved.
… Next step is to take specification to CR.

rakuco: If further development happened would it be Screen Wake Lock V2?

anssik: Easier: we can refresh the CR and only do a review for the delta.

marcosc: We have lots of options but we won't be able to get out of CR anyways until we have another implementation.
… We can have another CR when we have changes.

<marcosc> https://github.com/w3c/screen-wake-lock/issues/324

<marcosc> https://github.com/w3c/screen-wake-lock/pull/326

rakuco: Do we need to merge the outstanding pull request removing permissions checks.

marcosc: Yes, this one is important.

proposed RESOLUTION: Start the process to publish Screen Wake Lock API as a CR, check if the PR #326 triggers wide review for this delta

anssik: Would this change trigger another wide review?

marcosc: I don't think so.

[no objections to publish CR]

anssik: We can do a limited review on this PR, probably only affects privacy.

Action: Add a note mentioning future screen brightness work to the specification.

Resolution: Start the process to publish Screen Wake Lock API as a CR, check if the PR #326 triggers wide review for this delta

Interaction with Page Lifecycle

https://github.com/w3c/screen-wake-lock/issues/139

reilly: original questions was how Screen Wake Lock API interacts with Page Life Cycle
… since page lifecycle affects visibility, that's already exposed
… issue was open after last meeting to add informative examples how we acquire locks
… then reopen with a different question on doc being fully active
… marcosc correct me if doc can become fully active again?

marcosc: it can
… page hide event etc. can become fully active again if not destroyed

reillyg: the question is how much language the spec needs to handle becoming fully active?
… if the doc can in future become fully active then the event could fire
… "wait until the doc becomes active and then fire", is this prose needed?

marcosc: maybe it'd fire automatically if hidden, need to look at the implementation
… should the page fire events if it is frozen into back-forward cache?
… if page visibility handles this, then I'd be hesitant adding prose for this

marcosc: prefer closing this issue with a comment along the lines of this discussion
… I'll re-read the issue and have another look

reillyg: if you remove an iframe you lose visibility at the same time
… implementations may not fire visibilitychange in all cases

anssik: marcos to research WakeLock and Page Life Cycle #139 issue and report back in a follow-up meeting

Battery Status API

reillyg: Summarizing Issue #29

anssik: Was it Brave that chose to always return a static value?
… Are we in agreement that we want to make the recommendation stricter?

reillyg: We need to clean the text up.

anssik: No implementation uses a permission.

reillyg: There's a permission policy integration, but no prompt.

reillyg: An issue with the Geolocation API comes to mind. We are in a similar situation here.

Key words for use in RFCs to Indicate Requirement Levels

anssik: Do we have hooks for this?

Raphael: I don't think we do.

Action: Add normative text for private browsing mode behavior.

anssik: Are other APIs special-casing private browser mode?

Action: Remove reference to user permission as no implementation implements a permission prompt.

Raphael: Happens in implementation, but as far as I can tell not in specs

marcosc: Specs may have some "special mode" prose

reillyg: Idle Detection API has an example for this. We say that private browsing mode should be dealt with care when implementing this API.
… If there is a prompt, it should be automatically denied after a timeout.
… Battery Status should make all devices to appear plugged in and fully charged.

anssik: Presentation API did something similar.

marcosc: In the Tor browser, all activity can be considered private.
… You don't need to differentiate this.

anssik: It should be SHOULD in RFC language then.

Raphael: I understood reillyg was suggesting something stronger.

reillyg: Having this as guidance might be good, since private browsing mode is defined differently per UA
… Browsers should be reminded to make it not easy to detect

marcosc: It could be made an `if` statement in the steps
… If you are in private mode, return a new instance of battery and return this.

anssik: Do we need to create such a condition?

marcosc: We have this in Payments.

marcosc: As long as the condition is understood it might work

anssik: I remember using this for Vibration API.
… (Shows Note we have in Vibration API)

<marcosc> "Optionally, if the user agent wishes to disallow the call to show() to protect the user, then return a promise rejected with a "SecurityError" DOMException. For example, the user agent may limit the rate at which a page can call show(), as described in section § 14. Privacy and Security Considerations.

<marcosc> "

<xfq> W3C TAG Observations on Private Browsing Modes

marcosc: Yes, we do that for Payment Request

Vibrate "An implementation MAY return false and terminate these steps."

marcosc: It's a UA choice. There are reasons for this.

anssik: We acknowledge that this is important and that we converge to a solution
… Raphael has recently picked up speed with this spec.

Raphael: One option is to patch the security section, or to make it normative.

marcosc: Might be worth making it normative.

Rahpael: Would be a matter of not changing the initial values.

anssik: It could stay static right after the instance is created.

Raphael: If the UA doesn't want to report status.

anssik: We thought about this before.
… it's informative, though
… Raphael, maybe you come up with a proposal and seek review.

Raphael: OK

anssik: There is general agreement that we want to do something about this.
… Use issue #29 to submit feedback, and also from Samuel Weiler.

Raphael: Can anyone assign me the issue?

marcosc: Everyone in the WG should have write access to the repo. Checking.

anssik: marcosc will check permissions.

proposed RESOLUTION: Upgrade private browsing mode from MAY to SHOULD, integrate a hook into normative prose

Resolution: Upgrade private browsing mode from MAY to SHOULD, integrate a hook into normative prose

anssik: Raphael should now be able to assign himself

Make privacy mitigations normative

anssik: Made the change to make section 3 (formally 4) normative

anssik: We have worked on Generic Sensors, and have informative line items and matching line items
… We had to do this at the request of the privacy review

marcosc: You can always push back

anssik: From an implementer's PoV it's redundant, but not for privacy folks
… One option could be to allow for spec tooling to annotate this

marcosc: Potentially, but not sure if it could work.
… The HTML spec has the fingerprinting icon

<marcosc> https://www.w3.org/TR/payment-request/#privacy

anssik: I just want to acknowledge the privacy reviewers' work

anssik: We should make their work easier

marcosc: They might get misguided
… If the privacy and security section doesn't match the reality, that's bad
… Finding the balance is hard.

anssik: TAG is asking for an explainer

kenneth: Not sure if they were writing explainers

anssik: I want to raise this discussion with Sam, who is a leader in privacy
… Want to make our WG to be the most friendly to work with

anssik: It's a very important topic to me personally

Deprecation of non-SecureContext access to the getBattery()

Raphael: Everything is in shape

tomayac: YouTube let us know that they "will get all the data [they] need from that even if [they] no longer see data from embeds".

reillyg: The last comment was waiting for Tim, who is no longer working on Chrome

reilly: tim volodine no longer with Chrome project so don't block on his response

Raphael: Making slow progress

<marcosc> https://hacks.mozilla.org/2020/07/securing-gamepad-api/

marcosc: If it's not self-evident, we did something similar for the Gamepad API

tomayac: And Geolocation

Raphael: I will write an email to blink-dev and maybe write a blog post

tomayac: Offering to get a blog post out on this with Raphael on web.dev or developer.chrome.com

Network Information API

tomayac: And heavily recommending to be vocal about this.

tomayac: rebooting network information API.

<presentation>

tomayac: people have been confused about effective connection... "this is about technology, not about speed?"

tomayac: looking at caniuse.com, it's mostly red... Firefox has an outdated network info API

tomayac: nevertheless, it's a popular API - it's used on 42% of pages in the dataset

tomayac: overlaps with "save data API"

tomayac: there are privacy issues, and Mozilla considers it harmful... Mozilla would prefer declarative solutions. Apple also has privacy concerns. The TAG says (see presentation) ... There is an acknowledged use case.

tomayac: so, maybe it's time for a reboot!

tomayac: looking at the open issues - we should look at fixing effective connection and "metered"

Proposal for Network Information API reboot

tomayac: metered is supported on windows, where the user sets it explicitly

tomayac: this issue of "metered" is a concern around the world

tomayac: so, I trimmed the spec down to a simple event target, with `metered` and `sustainedSpeed`

tomayac: plus an onchange event

tomayac: use case - not downloading non essential content, like background sync

tomayac: another use case is limiting bandwidth usage, like with the Web Torrent network... when Torrent sees you have a lot of bandwidth, it will use as much as possible. So, this allows some restrictions to be applied (or adapt to).

tomayac: client hint is also part of the proposal... can send a hint if the connection is the metered and send the sustained speed

tomayac: this allows the server to adapt the experience ... like metered and non-metered experience

tomayac: I have some collected resources... I have a doc that collects some feedback from developers and users

tomayac: 80+ of users would be ok with sharing metered with websites

tomayac: we also recorded the different things that people would consider a good tradeoff for metered and sustained speed

tomayac: things like not playing videos, not loading heavy data things, etc.

tomayac: this lead to interesting discussion around sustainability and the environment

tomayac: someone also said they might use it for ads targeting...

Rebooting the Network Information API, motivational doc

tomayac: I asked Mozilla about their position

tomayac: I also reached out Apple

tomayac: but unfortunately, that didn't in a positive interaction

tomayac: asked Brave also, but didn't get any feedback yet

tomayac: I also looked at how implementable this is

tomayac: I also reached out to partners

tomayac: they were interested

tomayac: here is a link to the presentation

Slides: goo.gle/netinfo-reboot-slides

Motivation: goo.gle/netinfo-reboot-motivation

Explainer: goo.gle/netinfo-reboot-explainer

Spec: goo.gle/netinfo-reboot-spec

MC: what does onchange map to?

tomayac: it maps to both attributes... either changing

anssik: I heard that 40% of APIs are using the old API... is this new API backwards compatible?

DAS WG Tentative Deliverables: Network Information API

tomayac: yes, from what I've seen... a bunch of different big sites use it... youtube, tinder, facebook, etc. all use it. From my reverse engineering, what they do some analytics (so these changes may affect that). The content adaption that the current API provides would still be supported.

DAS WG Charter says "Note: the group will determine in collaboration with the WICG whether the existing implementations of this API on mobile warrant restarting the standardization process"

anssik: my question is, what is the temperature of WICG? are they interesting in continuing with this work?

WICG meta-issue for spec status discussion

tomayac: chats are happening across multiple working groups... could be web perf and us... or some privacy group that owns this. This is about reducing the finger printable area of the API.

anssik: this is great background info. And you shared it with browser vendors.

tomayac: yes, but it's not super positive... Mozilla says they can't really do sustained speed. And Apple are fairly against "metered" on privacy ground.

tomayac: I'd still like to reach out to Brave because they do have their own implementation (privacy preserving)

tomayac: what has worked so far is going through WICG...

marcosc: I mentioned previously this API is "poisoned" due to its history perhaps in the eyes of some folks
… renaming/refocusing the API might help get attention from more browser vendors
… put aside the API and look at use cases to solve, e.g. for videos, look at video use cases
… re CSS solution, it was palatable

tomayac: it seems like Chrome and Firefox engineering are disagreeing on this aspect, Chrome engineers feel the current methodology is accurate and works
… there is probably also misunderstanding on the level of accuracy, the new API uses bucketing to help with that
… Nielsen's Law of Internet Bandwidth tells us bandwidth grows 50% per year

marcosc: need to look if lazy loading could be implemented with other APIs

Use cases

Distinction from the save data use case

tomayac: I might need to get better at describing the use cases. There also very few, for instance, very limited number of web apps that would make use of this

reillyg: I think we're in theory in an excellent position to document the use cases given 40% page load use the existing API
… we want to understand why the API is so popular
… arguing on the usefulness based on existing app usage is a reasonable approach

reillyg: at least we are in a really good position to document the use cases, given the large usage... we can evaluate the utility based on real world, given that 40% of sites are using it. We can form a good argument based on what we find. We mentioned the video use case - we should get this in front of the media folks.
… for video use cases, we should go to folks working on web video spec features and talk to large video web sites

MC: "Review of apps that use network information" for native app usage - https://www.w3.org/TR/netinfo-usecases/ 😇

marcosc: for native apps, I looked 20-ish apps in 2014
… you can find that information helpful, there's a doc published

Geolocation API

Republish as CR #93

Explicitly limit permission lifetimes

marcosc: REC publication is currently blocked on issue #69, marcosc to handle that

HTML Media Capture

Development transferred to WHATWG

anssik: just announcing the feature is moving to HTML LS

Summary of action items

  1. alexis to add a note for possible future work to support new form factors
  2. diego to open a GH issue to collect developer feedback on whether they want to differentiate pen input from touch input
  3. Add a note mentioning future screen brightness work to the specification.
  4. Add normative text for private browsing mode behavior.
  5. Remove reference to user permission as no implementation implements a permission prompt.

Summary of resolutions

  1. Start the process to publish Screen Wake Lock API as a CR, check if the PR #326 triggers wide review for this delta
  2. Upgrade private browsing mode from MAY to SHOULD, integrate a hook into normative prose
Minutes manually created (not a transcript), formatted by scribe.perl version 136 (Thu May 27 13:50:24 2021 UTC).

Diagnostics

Succeeded: s/agnel/angle

Succeeded: s/ACTION alexis to add a note for possible future work to support new form factors//

Succeeded: s/background/back-forward

Succeeded: s/Reading summarizing/Summarizing

Succeeded: s/into mind/to mind

Succeeded: s/no longer informative/normative

Succeeded: s/their work/the privacy reviewers' work

Succeeded: s/Depreciation/Deprecation/

Maybe present: Alexis, anssik, diekus, Explainer, Kenneth, Laura, Laura_M, marcosc, MC, Motivation, Rahpael, rakuco, Raphael, reilly, reillyg, Slides, Spec, tomayac