Meeting minutes
<amandy> cannot join zoom either
<amandy> An error occurred. Sorry, the page you are looking for is currently unavailable. Please try again later.
<willmorgan> HTTP 502 for me
<amandy> same here
<willmorgan> ah, I am not chair, I am human
<tomayac> Devices and Sensors
<tomayac> Thursday, September 15 · 08:00 – 08:30
<tomayac> Google Meet joining info
<tomayac> Video call link: https://
<tomayac> Or dial: (CA) +1 437-781-4585 PIN: 595 646 086 5026#
<tomayac> More phone numbers: https://
Welcome
ghurlbot, this is w3c/devicesensors-wg
<ghurlbot> anssik, OK
anssik: Welcome to the Devices and Sensors WG TPAC 2022 meeting
… a few practical things:
Code of Ethics and Professional Conduct
… To speak up we use a queue on IRC: q+, q-
… See also TPAC 2022 site for the latest information
Intros
anssik: Anssi Kostiainen, DAS WG co-chair, Intel
<fbeaufort> FYI mit.zoom.us seems to be back.
join Zoom https://
<bneedham> trying to reset my camera from switching
Thomas Steiner, Google DevDel, Project Fugu
<scheib> Vincent Scheib from Google, working on Capabilities / Fugu ... device APIs, installable desktop PWAs, supporting web audio
Reilly Grant, WG co-chair, TL for Device APIs team and Isolated Web Apps at Google
Philippe Le Hegaret, PM of W3C
<cmartin> Chris Martin from Mozilla -- Device Interfaces and low-level OS integration for Windows (only here as an observer)
Ralph Swick W3C
Arno Mandy, Intel, Generic Sensors, Compute Pressure implementation in Chromium
Bradley Needham, Sony Playstation, extension to Gamepad spec and impl for DS4
Francois Beaufort, Google DevRel, love Device API and guitars!!
Fuqiao Xue: W3C Staff contact for this WG
Kenneth Christiansen: Web Platform team Intel, working on Compute Pressure
Raphael Kubo da Costa, Intel, Screen Wake Lock, Sensors, Compute Pressure, janitor of old specs in this WG
Will Morgan, head of web innocation iProov, Screen brightness work is our interest, also ALS, Generic Sensor work
Alexis Menard, Intel, involved with Device Posture API mostly
Charter update
Update on the proposed charter in review
Status of Formal Objections (Member-only)
anssik: I invited plh to give us an update on the FO
… we're switching TBL out from the process, so some pains there
[plh shares the FO planning document]
plh: I've been working on the report prepared for DAS to be sent to the AB+TAG Council
… you're the first people to see it
… FO from Mozilla, sent to public record also
… two disagreements about 1) existing deliverables 2) new deliverables
<Zakim> plh, you wanted to ask about TAG reviews
https://
https://
https://
reillyg: I want to mention that in the previous meeting we agreed on a plan to review those APIs that have most implementer interest
… this includes Geolocation API that went to Rec recently
… I'm proposing we should perhaps next ask TAG review the DeviceOrientation API
… we should be careful in asking reviews, ask reviews one by one
plh: it is important to fill in the report with this new information
… some people will be coming from outside the WG and lock the context
rakuco: questions about TAG reviews, should they be requested periodically?
plh: back in Dec 2020 Council asked the WG to do TAG reviews, that we need to answers
rakuco: looking at the doc, not clear what chairs vs. regular contributors should do?
… we need to make sure the history of the WG is well captured
… the doc will be presented to individuals who have not followed the work
tomayac: does demonstrated user demand help here?
plh: deployed and shipped?
… you're welcome to write that [in the report?]
willmorgan: I'm in a position to deploy an app that is reaching millions of users a week
… using ALS, Device Orientation, Screen Brightness
… if you look for concrete implementation, my company and a number of other companies would be willing to go on record on our usage
plh: no need to make a case for the use case
willmorgan: is these anything else could be lacking except TAG review?
plh: are we solving the problems the right way?
tomayac: as Anssi pointed our, Screen Brightness is trending in WebKit positions, use case being validated by a number of vendors
… there have been several proposals for this particular API, when is the right moment to do a TAG review?
<willmorgan> As an example for screen brightness, DAS went to TC39 and CSS WG to ensure that we're actually solving the use case in the best way. We also have multiple current and ex members of TAG. So when is the right moment for TAG review? Probably when there is enough consensus in and out of the the DAS WG?
2023-2024 scope discussion, Low level Device APIs breakout session
anssik: our current charter ends 2022-11-30
… so regardless of the Formal Objection status we must recharter again for the next two-year period 2022-12-01 - 2024-11-30
plh: or short extension
anssik: as usual, we review the device APIs landscape for promising new incubations
[DRAFT] Devices and Sensors Working Group Charter
anssik: we collect feedback for charter through the GH repo
DAS WG Charter GH issues
… looking at the issue from most recently updated first
… it is suggested the Compute Pressure API scope to be adjusted based on feedback from user's, to explicitly add in scope "v2 use cases"
Compute Pressure API v2 use cases
DAS WG Charter: Compute Pressure API
reillyg: maybe it is worth discussing the contrary option, we have a process in W3C to incubate in WICG
reillyg: looking at new work, should we consider the Council's recommendations, being more critical of the scope
… waiting for multi-implementer interest before expanding the scope
… ability of the WG to advance its specs is hindered by other implementer's unwillingness to contribute to the WG
anssik: this WG is following the W3C Process and that doc does not require multi-implementer commitment for WG to take on new work
plh: this is a hard issue
<plh>
reillyg: I want to understand what is needed to help people join us
scheib: people have different motivations, but we're practical about it, if ppl don't want to collaborate we still want to solve the problems people have
… incubation is healthy, we shouldn't be too attached where the work happens
scheib: willmorgan wants to see a solution in place and make progress on it
plh: W3C Process is here to validate your work
… in an ideal world there would be no process
… the process is to say, this is how we validate the work: horizontal review, TAG review etc.
… you can do that outside the W3C Process of course
scheib: W3C is facilitating the work
plh: that is perfectly fine, I want to make sure you make the right decision
willmorgan: on getting other implementers on board, I don't have interest in philosophy of each implementer
… this is the only group where philosophical differences are noticeable
… I'm curious where people think the web should be going
… toward a platform capable of doing many things?
… there's a clear generic use case, sometimes you need to do powerful things but don't want to download an app, and DAS WG covers that use case, this is important
<willmorgan> sorry - do not mean to stall progress
diego: this reminds me of a discussion with some Samsung folks, Device Posture API
… we had two workstreams, Intel enabling the API on Windows, and Samsung working on Android
… the questions was, are these two implementations sufficiently different to consider two implementations?
… codebases for different platforms might be considered different implementations
anssik: this is a great perspective, I've also raised this, also another datapoint is the shared WebRTC library across all major implementations
reillyg: I'm leaning towards keeping the scope as is
<plh> [thanks to the Working Group]
Privacy & Security
Sam: you're doing great! Appreciate your contributions and collaboration
… device discovery is one generic privacy issue
… having a device available adds one bit of entropy
… you know about specific vulnerabilities
… working relationship with you is great, my colleagues feel the same
anssik: thanks Sam for your kind words!
Sam: sometimes we're [PING] not great in other things that "here's the whole doc review"
<Zakim> rakuco, you wanted to talk about my impressions
rakuco: I've been landing and/or reviewing PRs to some of our specs since last year's TPAC
… Generic Sensor, ALS, Battery
… I think my main take away is that privacy work is harder than implementing them
… folks have less time for privacy patch review
… I've learned best practices on the go reading PING documentation
… in general making focus improvements is easier
… I'm not sure how wide reviews for our specs would entail
… some involving permissions is hard to reach consensus on, when we touch upper layers of the implementation (UI/UX?)
… what criteria to use to make a decision on those issues
… sometimes PING guidance is not always clear, I couldn't find a source how to calculate the entropy mentioned in some of the docs
… also ephemeral fingerprinting did not have a good reference, there was a 2-year old repo for that
… looking ahead, what we could do to improve the process, speed it up, should we redo questionnaires?
Sam: we pay a lot of attention to device discovery due to fingerprinting concerns
… look at ways of shifting device picking into the browser, if the user needs to select something delegate that to the browser (UI)
Sam: note what we are opening up, a lot of this is domain-specific so you're the experts
… when and how I make this available
reillyg: my question ducktails on the ephemeral fingerprinting
… the sensor issues around accuracy and attach based on sensor data, there's a fundamental issue: a device does or not have a gyroscope/hw capability
… in the audio space, we have path forward to avoiding more bits
… at some point you get down to a boolean question like sensor availability
… what is the model for providing that type of information
… there bits add up
Sam: example from audio, explaining sample rate of devices, there was no pattern for this
… specialized hardware that use unusual sampling rate, do not expose that information
… how can we make all the devices look the same even when they are not?
tomayac: that makes some use cases complicated? e.g. I'm telling I'm a 4K camera, even if I'm not
Sam: we want the default to look the same
anssik: we use static defaults for Battery Status API
alexis_menard: you may not have 3 gyros in your device, in foldables there actually are multiple gyros
… I'm not sure if fake gyro works
<Zakim> weiler, you wanted to discuss permissions workshop December 5-6, Munich
weiler: there's likely to be a Permissions workshop in Munich in December 2022
… mark your calendars DAS WG participants
<willmorgan> count me in...
<weiler> https://
willmorgan: if we were to start providing fuzzy info for gyros it might break a lot of existing stuff
anssik: we don't want to break backwards compat
rakuco: I'm trying to think of a concrete example on what Sam proposed
… if we call Accelerometer.start() it'll fail with a specific error when there's no accelerometer
… so given the PING guidance, we wouldn't return anything at all?
Sam: we'd want to return the same error if there's no accelerometer or no permission to use it
reillyg: in the current model, WebKit is the only engine that implements permissions for DeviceOrientation
… there were two options: "denied" & "provide no data" or provide low resolution data
rakuco: wanted to clarify Generic Sensors example: would it be better to provide fake data for accelerometer that does not exists or throw an error?
Sam: no guideline, need to be figured on on a use case basis
<willmorgan> I think absent accelerometer returns as 0, 0, 0 in motion events
reillyg: if we could fake data when no permissions are available, once granted
… would we also want to provide fingerprinting protection
Sam: I'd say look at layers, there are things you can do, even if you grant permissions you can get sample rates (in Web Audio API)
… we can't sample everyone, but we can constrain this much
<fbeaufort> Marcos is in the Immersive Group right now and will attend the TPAC Devices and Sensors WG meeting after lunch. He asks me if we can move "Screen Wake Lock API" item so that he can be there.
Sam: even if the user has granted permission, minimize fingerprint still consistent with the environment and use case
Sensor Framework
ghurlbot, this is w3c/sensors
<ghurlbot> anssik, OK
#429
anssik: related to the previous discussions, quantization and threshold check algorithms for Generic Sensor API received PING review and finally landed after an extensive 6 month review
rakuco: good news here, after a lot of work this has landed
… our ALS implementation in Chrome had some issues and we implemented mitigations, then also brought those changes to the spec so other implementers can benefit from them
… reviewed with PING and Lukasz, IE in this WG and author of paper describing the attack we mitigate against with this PR
… as for next steps, we could extend this for rounding for motion sensors we do in Chrome
… we do rounding for most sensors, but only spec it in ALS spec
reillyg: we do that in Device Orientation spec, but did not bring those mitigation back into the spec
anssik: any concerns?
[no concerns]
RESOLUTION: Incorporate quantization measures from Device Orientation into motion sensor specs (accelerometer, gyroscope, device orientation)
anssik: no other bugs for the Generic Sensor API, we're doing great it seems!
Motion Sensors
anssik: For completeness, this agenda enumerates all the WG's deliverables, including tentative ones.
… I want to check if anyone is aware of issues we should discuss for any of the Motion Sensors marked with "none". "None" indicates no issues proposed to the F2F agenda currently.
anssik: anything we missed in GH bug triage?
reillyg: checked Chrome metrics for these, the Sensor APIs we shipped are duplicates of Device Orientation because they provide a better API for developers, allow configurability and modern API surface
… most usage is on old Device Orientation, likely due to the fact developers use what works across browsers
<kenneth> https://
<kenneth> 579 weekly downloads
anssik: the polyfill could help raise awareness among developers
<kenneth> https://
<kenneth> to compare with another polyfill
kenneth: when I talk to web developers, many don't know these features are available
reillyg: something I've been thinking recently, I feel there used to be a lot more interest in Sensor APIs when XR/AR used sensors a lot
… it feels like WebXR has moved to its own API from the polyfill
tomayac: dated spec URLs were an issue, now web.dev has a landing page for Generic Sensors
willmorgan: from an implementer's PoV, I'm aware this is changing, why we haven't migrated yet is it is literally just easier to deal with the old events listening on window
willmorgan: if the polyfill could make the usage easier with permissions then it'd be ideal
<kenneth> we accept patches to the polyfill :-) I will gladly review
<willmorgan> I will pass this through to the dev working on this at the mo, thanks Kenneth!
Accelerometer
[no f2f issues]
Gyroscope
[no f2f issues]
Magnetometer
[no f2f issues]
Orientation Sensor
[no f2f issues]
Device Orientation
ghurlbot, this is w3c/deviceorientation
<ghurlbot> anssik, OK
#88
<ghurlbot> Pull Request 88 Drop duplicate PermissionState definition (foolip)
anssik: Drop duplicate PermissionState definition PR has been open for long so I put it on the agenda to see if we can nudge it forward
… the concrete spec change is small, removes:
enum PermissionState {
"granted",
"denied",
};
anssik: but the devil is in the details, who wants to take a stab at the status?
rakuco: IIRC, the more general problem is this API does not integrate with the permission spec
… we duplicate some Permissions spec text in Device Orientation
… not sure if removing this would break WebKit
… need to think of powerful feature names
… WebKit checks these strings "accelerometer", "gyroscope", and "magnetometer" for Permissions Policy
willmorgan: just to confirm, the permissions policy gives the JS code within an origin a permission to request a permission from the user?
… if you don't have it on your permissions policy, any calls will fulfill with denied?
… for context, this causes problems for shipping code for people who iframe your apps
… that's my 2cent, it's confusing
reillyg: the error message should be different
… not allowed error
… this is a bit separate, confusingly, permissions policy and permissions are unrelated features, not integrated at all
… we should eventually resolve this issue
<willmorgan> sorry!!!
reillyg: originally named as "Feature Policy", then renamed and split into two "Permissions Policy" and "Document Policy"
reillyg: the theory was, the things in FP that were features "how should this HTML work with X" would go to Document Policy
… things related to permissions go to Permissions Policy
reillyg: I think that we should eventually fully integrate with the Permissions spec
… similarly to what Generic Sensor API does
… for this issue #88 the only issue is the algorithms text still referred to an enum value that was no longer defined
<ghurlbot> Pull Request 88 Drop duplicate PermissionState definition (foolip)
reillyg: we should refer to the Permissions spec enum value instead
rakuco: I haven't look at this in a while
rakuco: if we just integrate with Permissions spec, would we break WebKit?
reillyg: next topic is how to take this to Rec
reillyg: the API returns a custom enum for backwards compat (with WebKit) reasons
anssik: was this a namespace clash issue?
reillyg: correct, we could have a mapping table from what is shipping in WebKit to Permissions spec enum values
rakuco: was there a behaviour difference too?
reillyg: algorithm seems like the same
anssik: do we need to ask WebKit implementation to change?
rakuco: it's not possible to return default, but default does not act like prompt exactly
reillyg: the issue is in Permissions spec integration in many places, for user gesture concept, many specs need some custom steps
… including Generic Sensor API with custom user activation check in place
… we should upstream it to Permissions API but that is not a blocker IMO
reillyg: maybe rakuco will take a look, and we try to land this at the earliest opportunity
<ghurlbot> Pull Request 88 Drop duplicate PermissionState definition (foolip)
RESOLUTION: take another look at issue #88 Drop duplicate PermissionState definition and provide feedback on what still needs to be done to land
reillyg: Now that the Geolocation API has reached Recommendation status should we start the process of doing the same for this specification?
… we care about spec maintenance, so I'd propose the next legacy API is Device Orientation to move to CR and toward Rec
… request TAG review and other horizontal reviews
reillyg: privacy mitigation done, would be great to get that stamped
anssik: I agree, do we have an explainer for this API that is a requirement for Device Orientation?
reillyg: I think an extensive introduction can almost serve as an explainer
… pull text from intro to an explainer?
anssik: any concerns with the WG starting to move Device Orientation toward CR and Rec?
rakuco: couple of questions about the spec itself
?
rakuco: w-p-t, a lot of test for Device Orientation spec but many of those only pass in Chrome
anssik: what is missing in other engines?
rakuco: they don't implement the same mocks?
rakuco: optimally, we'd use WebDriver so other engines could interop, now we rely on JS mocks in Blink
reillyg: similar concern raised for Geolocation
… Chrome DevTools can fake Geolocation, there is no similar support for DeviceOrientation
… we have WebDriver spec'd but not implemented
… it'd be reasonable to say, we need to figure out multi-implementer i) permissions and ii) automated testing approach, not rely on manual testing
scheib: Device Orientation, I read the introduction section to size it how good an explainer text this is -- it's primarily missing the "why"
… it simply states what it is, now "why" it is
… maybe not a blocker, but would probably get friction if this was a new proposal
… second point, we have a superior modern API based on Generic Sensor API, but we want to take the old API to Rec?
… how much energy we want to put on the old spec or simply say it is the new one we want to proceed with?
reillyg: this is why we did the Geolocation API Rec, we need to recognize we're maintaining old legacy specs that are being used
… we're not adding new features, but we need to integrate with permissions if that is what developers need
<scheib> Links:
<scheib> https://
<scheib> https://
<scheib> https://
https://
scheib: I see a component missing, why do we need orientation at all?
<scheib> use cases in https://
https://
<willmorgan> iProov is using it for liveness detection as well
scheib: I still think we're insufficiently motivating the problem, more and more W3C tries to work with other implementers, explain the problem first, then propose the solution
… this should be explained in the explainer document
reillyg: in addition to the immersive experiences and gaming use cases, we should acknowledge a lot of these use cases are motivated by fraud detection
willmorgan: we'd look to use the sensors with trust tokens
willmorgan: how long sensors are granted, if the use case does not need them more than for a minute, that is one possible privacy issue mitigation
scheib: proposal to clean up the old spec and direct to the new spec for introduction and motivation, driving to the modern API
RESOLUTION: Start advancing the DeviceOrientation Event spec toward Rec, produce an explainer, and evaluate areas of concern prior to wide review
msw: I found the Device Orientation spec when I was looking at Screen Orientation spec that exposes an angle attribute
… that is inaccurate, presumably done to workaround the new spec, that should be the lightly lit path for anyone using the screen orientation angle
reillyg: the description is inaccurate?
msw: what I've seen from the wpt, implementation are unreliable
anssik: Screen Orientation exposes type of orientation, discreet states
msw: it also exposes a numeric angle
… Device Orientation would be a replacement for the numeric angle?
reillyg: difference between Screen Orientation and Device Orientation: Screen exposes the landscape, portrait etc. orientation, similar to Device Posture
… semantics vs. physical
<msw> sorry for raising a topic from a place of ignorance! :-/
Device Capabilities
Battery Status API
ghurlbot, this is w3c/battery
<ghurlbot> anssik, OK
#52
<ghurlbot> Issue 52 Explore a high-level API (anssiko)
anssik: Chris Palmer provided a number of good use cases for a high-level API in a crbug https://
… summarizing: "I believe these valid use-cases can be satisfied with `onlowpower` and maybe `onfullpower` events, rather than the current API."
anssik: Francois has produced an elegant high-level API polyfill in 8 lines of JS
… my question is do folks like this high-level API? any obvious issues?
… and if so, do we have a good migration path for the existing API to the new API?
… the implementations of the existing API could aggressively round the readouts to simulate the high-level API already today, report two discrete values e.g. 0.09 and 1.0 mapping to "lowpower" and "fullpower"
tomayac: for native apps, power saving mode kicks in at some point and otherwise apps don't care
kenneth, there's an issue for that https://
reillyg: I think the battery saving mode idea is that it shifts the power mgmt policy, e.g. timer behaviour, it forces apps to use less power
tomayac: does it disable bg sync?
reillyg: yes
… I'd like to understand the existing use cases of the battery status API?
use cases in https://
reillyg: those are good use cases, I'd like to see that real sites are using the API this way
… we should look at other platforms for examples of apps that adjust their behavior based on battery status level
… we talked to YT embed, they were using this API to switch to a lower resolution video format
scheib: there's a lot of usage for this API
reillyg: similar to device orientation and motion
scheib: on Chrome this is 10% of page loads
<scheib> https://
tomayac: YT is using this API
reillyg: I like the "lowbattery" proposal
<kenneth> constrained battery :-)
<kenneth> I would like my webapps to preserve battery when I manually put on battery saver mode
jsbell: regarding high-level API, we'd love if another implementer goes first
rakuco: I was also part of this investigation, to see if we can stop exposing BatteryStatus in secure context, there were various usages in http websites but did not see sites changing their behaviour
… could these be signals in Compute Pressure API?
anssik: it would help us make a decision on this API and the proposed high-level API if we'd know what the usage is now
#54
<ghurlbot> Issue 54 Use of "current global object" is confusing (Ms2ger)
willmorgan: if OS is throttling would be useful to know
anssik: there's Compute Pressure API for OS throttling related use cases
RESOLUTION: Understand the use cases behind current usage of the API, and if compelling, seek positions from other implementers for interest for the high-level Battery proposal
Device Posture API
ghurlbot, this is w3c/device-posture
<ghurlbot> anssik, OK
#100
<ghurlbot> Pull Request 100 Remove Folded Over from Device Posture (supo2co)
anssik: proposal to remove "folded-over" motivated by experiences from the WindowManager Jetpack library for Android
https://
alexis_menard: they removed "folded over posture"
… to do content side-by-side with other content needs this
… we have devices with foldable screens that can do 360
… not against removing, we can add in the future
kenneth: if no implementation, agree to remove
Dongwoo: I'm from Samsung, worked with Diago when he worked on this API
… Samsung also does not have "folded the other way" devices
… Huawei has some devices that fold like that
… "folded" and "continuous" is enough for our purposes
reillyg: "folded-over" posture, are there not devices that support that posture?
… Android removed it because foldable OLEDs do not support that
alexis_menard: technically the devices can support "folded-over" posture but there's no platform API to get this information on Windows
Dongwoo: is there any plan from different browser vendors to support this API?
… Samsung supports this API and why try to connect service providers to use this API, e.g. in developer conference next month we do a presentation with one of our partners to show live web page using this API and Viewport Media Query
alexis_menard: Chromium does not depend on JetPack library
… on Windows side, there's no official API for posture information, the best is hinge angle from the SDK
alexis_menard: two 2nd gen new foldable devices coming to the market, this is public information, getting this API working would be great
Chris_Martin: there's a lot of entropy in the previous iteration of the API
alexis_menard: we have mitigated fingerprinting satisfactorily in the latest API by reduced entropy
Chris_Martin: is cross-tab correlation possible?
reillyg: exposed to visible content only as a mitigation
alexis_menard: we don't give info how big the fold angle is, it is "folded" or "not folder", minimizing the fingerprinting concern
… only visible and in focus pages get these events
RESOLUTION: Remove "Folded Over" posture from Device Posture API due to platform API limitations
Environment Sensors
Ambient Light Sensor
ghurlbot, this is w3c/ambient-light
<ghurlbot> anssik, OK
#77
rakuco: we're working on wpt tests for these changes
#79
<ghurlbot> Issue 79 Add camera permission requirement to spec? (rakuco)
rakuco: last year we discussed this and made a resolution to add camera permission to ALS spec
… some time passed and I looked at this and had questions and was not sure how to do this if we decide to do this
reillyg: to summarize, ALS is in front of the phone to detect the ambient light, used by camera to also measure ambient light for better exposure adjustment, also used for iProov use case how light is reflected off of environment to understand if the person is real
… ALS exposes information about the user's environment, so concerns over side channel attacks
… this is different from motion sensors
… use cases involve camera with ALS, so proposal was to grant ALS access if camera has already access
… ALS is in practice a camera with a resolution of 1x1 pixels :)
rakuco: the problem is how do you prompt the user for access if they don't know what light sensor is
reillyg: apps could have asked for camera access already
rakuco: some concerns may be UA concerns, and we try to solve this on a spec level
… my concern is if we risk changing the constraints in the getUserMedia, how to integrate with media capture permission management
… hypothetical device with camera and not ALS
larsgk: there was a case at Maersk, we wanted to use this on a vessel, for day, night, dusk, in this case tying to a camera permission makes no sense
in that environment granting access to camera is not appropriate
willmorgan: side channel is mitigated by quantization and rounding?
reillyg: good questions, I don't recall what the states reason was for moving motions sensors from being granted by default and say ALS would be OK
Theodore_Olsauskas-Warren: trying to the camera permission is a good approach, but has few failure scenarios: site that only wants ALS, but not camera
… WebXR has a similar approach with its depth map vs. regular camera permission
… do users understand what ALS means, use cases are specialized
… I like bundling permissions
reillyg: explaining sensors is difficult, we say "Detect the motion of your device" rather than "Access sensors"
willmorgan: in my use where it is bundling powerful browser APIs, we may be asking ALS depending on how it is implemented
… easier to have context where we see permissions going from WebAppSec WG, are they allowing more of bundling permissions, otherwise we end up doing weird things like bundling with camera
… you may need a virtual video device to make this happen
… I'd be happy if we piggy-pack on the camera permission and tolerate we cannot go the other way around
<fbeaufort> FYI Marcos is coming in 15mins
reillyg: in spec discussion we identified ALS with its own permission request method, requesting a camera permission
FTR: Chris_Martin in above ALS discussion is Theodore_Olsauskas-Warren
reillyg: it looks like the proposal to integrate with camera permission is the most appropriate approach
rakuco: Should we still require camera permission?
… or just assume it's there?
reillyg: It would be better to request, because that avoids the situation that Theodore_Olsauskas-Warren mentioned
rakuco: Should ALS be listed in media enumeration?
reillyg: If we're only requiring it, then it would not appear
willmorgan: Firefox will prompt you for the device. If you only have one, not sure what happens.
reillyg: What would happen if it requested camera?
willmorgan: It would probably be not that bad
… You could customize the text or display a dropdown
rakuco: Should stopping the camera stop the ALS?
reillyg: It should stop the indicator and activate it like the camera does.
willmorgan: If you close the media stream, does that stop the sensor?
… Is there a mechanism within the existing API to stop the feed without the developer explicitly asking to stop?
reillyg: We could throw an exception "camera ended"
… This would tie them even closer together.
willmorgan: It would probably be cleaner to just drop this off.
… You would stop the media stream together.
rakuco: Should we only tie the permissions together, or tie the sensor to a stream? Would there be a way to only have the indicator, without the actual camera being on?
reillyg: It makes sense from a user perspective
… It would make it so the physical indicator would be in sync with the ALS
Theodore_Olsauskas-Warren: Are there a lot of use cases that only require ALS?
… Or are the use cases very specific, and operators on a ship for example can understand that it's not really the camera
reillyg: Seems reasonable to request camera and ALS together
Theodore_Olsauskas-Warren: Does this affect the use cases?
reillyg: Every device that has fulfills the use cases also has a camera. Maybe old laptops to activate the keyboard lights.
willmorgan: We would not bother to ask.
larsgk: I would get this through if it were easier.
… What about a car dashboard?
… Could there be a media query instead?
… Maybe car dashboards?
… With maybe two or three coarse values?
https://
reillyg: Similar to light mode dark mode, a media query for ambient light might be useful, albeit we haven't seen it
Theodore_Olsauskas-Warren: Is this for regular browsers, or special environments?
larsgk: Google Maps switches when you go in a tunnel
reillyg: Talking with delivery drivers, we know that this exists
anssik: You can follow up in the issue discussion if you're interested
… I might be wrong, but I think it was hard to reach interop
anssik: I believe it was hard to define what dusk etc. means
rakuco: There used to be a light level media query, but it was dropped.
… Because it could achieve the same as prefers-contrast
… We seem to tie ALS and camera close together
willmorgan: I have to check the thread, but it was long and caused me to flipflop my own opinion.
… I'll be talking about providing a media stream of a 1x1 pixel camera
rakuco: This is what we proposed
willmorgan: I don't know enough about the inner workings of an ALS
willmorgan: I can't think of a use case where you would want this
reillyg: The units would be different
… An ALS provides a more calibrated light level in lumens, so the camera can be calibrated
rakuco: We would need to turn a camera on to get a media stream
/me PROPOSED RESOLUTION: Prepare a specification proposal for coupling the ALS to an active media stream rather than camera permission.
RESOLUTION: Prepare a specification proposal for coupling the ALS to an active media stream rather than camera permission.
rakuco: Would this be basically creating a new stream or track?
reillyg: The existing stream doesn't make sense for the format. It would need to be a special track
willmorgan: We would need to tie it to an active media stream
rakuco: ALS would not be creating this stream, just be attached to it.
tomayac: If there's no camera, you cannot engage this stream, right?
willmorgan: You can create a virtual camera in such cases. Horrible, but works.
Device Capabilities (cont'd)
Screen Brightness API
ghurlbot, this is w3c/screen-wake-lock
<ghurlbot> anssik, OK
reillyg: Screen brightness, we have the brightness mode explainer
… And we opened a standards position thread
… Which elicited some interesting feedback
<willmorgan> https://
reillyg: Let's reaffirm the use cases we're trying to resolve
… As I understand it: the generic use case is the one where the developer wants to make a section or the entire screen brighter, so a QR code could be read
… We proposed a couple of actions
… Either an explicit brightness API, or as a part of the fullscreen API
… Or a declarative API
… The user agent can then decide whether it wants to adhere or not
… The use case is still scanning
… Are there concerns about the use cases per se?
Marcosc: If we disassociate the use cases from the API shape, both on iOS and Android, you're either using the app, or iOS Wallet
reillyg: Interesting that it only boosts the brightness of the code
Marcosc: At least on the Apple side, we have a system component for doing this
… If you want to have a boarding pass, you download it
reillyg: Is there an open standard?
Marcosc: No, but there is a format
… This tech exists, and this is what Apple does today
… For the authentication case, I did see the Australian government thing with the flashing and all.
… It's not about web authentication as an API, only about the use case
… Brightness helps, but more sophisticated sensor on an iPhone exist. Like Face ID.
… Taking a step back. Let's find an existing usable thing, maybe at the OS level
reillyg: My understanding is that this is not something that could be delegated to the OS
… Face ID is not solving the usecase of iProov
willmorgan: A couple of points on QR codes. Web Authn is brilliant, however, it ties the authentication to a device
… I have two laptops, but I can only bind one security key to a device
… The second issue is how do I establish trust in a device, before I bind the device
… It's funny that you mention Face ID. Getting access to that sensor would be brilliant. Or to infrared cameras.
… Using some of the same sensors the OS is using would be great
… For now, my use case is a complement to WebAuthn
… We (iProov) encourage WebAuthn
… WebAuthn is not a replacement
… For QR stuff, I have passes, but I don't want an app for that
… I feel like Wallet doesn't completely replace the QR code case
Marcosc: I live close to a post office, having the QR code a click away from the SMS for example is usefuk
]s/useful/useful
reillyg: It seemed like the direction from Apple was more to declaratively solve this, like with CSS
… You could mark a region and say in CSS that this needs boosting
… How would the user agent react to this. It gives the user agent a lot of latitude
… The user agent can detect if there's really a QR code, and if so, boost it
… It's in the pattern to give the user agent more information about what is on the page
Marcosc: Reiterating what you're saying
… You scroll down, and author says this is brightable, the user agent could then verify it is a QR code, and then boost the region
… This would solve the Wikipedia article case, since the regions would not be marked as boosted
reillyg: The iProov use case would be solved by this
… This would improve upon the status quo where the user has to be told to open quick settings
… I think making that declarative is more manageable, since the user agent would have more context
Marcosc: And it doesn't leak information
reillyg: Counters abuse
… What the WG should do is to focus the conversation on the declarative proposal
… and see what CSS WG think about it
Marcosc: It could be an element attribute as well
reillyg: We would then bring this back to the WebKit repo
fbeaufort: How is this different from the fullscreen proposal?
Marcosc: They are not mutually exclusive
… You might still go fullscreen and request boosted brightness
… It could be paired with fullscreen, like when you hand over a movie ticket
Marcosc: We haven't solved the way how to get out of fullscreen, which is why we haven't done it on Safari
Marcosc: Fullscreen comes with nice things
reillyg: I don't think we need both, but including brightness in fullscreen makes sense.
willmorgan: iPadOS does support fullscreen, iOS doesn't
… Going back to the fullscreen API, this would help my use case
scheib: Are the nuances of the use cases communicated correctly to Marcosc?
… Most of what I heard was focused on the QR code use case.
Marcosc: We're discussing this actively
willmorgan: It's a complicated thing. What we're trying to prove is that the thing we're pointing the camera at is 3D.
Marcosc: Is this about making sure it's the same person?
willmorgan: Feature matching is one thing we do.
Marcosc: If you had the assurance of the infrared sensor, this would work
willmorgan: We're currently using depth data
willmorgan: I know on Surface laptops you can get access to infrared cameras
… This is not about fingerprinting users
Marcosc: What if we had a "compare A to B" service?
… So that the matching would be done on the device
willmorgan: Maybe. It would be only one of the signals.
Marcosc: If I log in with Passkeys, it works
willmorgan: We're doing stuff on device. It's a limited use case.
willmorgan: There are also compliance reasons, people in the banks would like to know that someone was actually a person, and not just get a hash
… This is a huge industry.
Marcosc: I think you have something quite interesting, and brightness is a small part of it.
… If we take the larger part, what would be the technology to solve your use case
Marcosc: What is the ultimate way to solve this problem.
msw: Related to that idea, alternative methods would include to move the head to a set of poses
… It would come up with a set of sequences
willmorgan: We have that already. This problem falls for deep fakes
msw: Understood. Is it not possible to fake light changes.
reillyg: I have seen a demo of lighting changes
reillyg: The conclusion from the last message on the thread from Marcosc was to discuss in person
reillyg: Now you have collected a lot of info and the understanding is that there will be a response
Marcosc: I will take back those declarative proposals
fbeaufort: Talking about the declarative approach, the CSS property was part of the explainer
… Did WebKit not address this in their response?
https://
reillyg: The issue is less the API, but the use cases
… Now WebKit can make a more informed decision
Marcosc: I learned Safari detects QR codes, but doesn't currently do anything with this info
reillyg: Up next would be Screen Wake Lock and removing the permission requirements
… Then maybe system wake lock
… Then some other things we can address
Screen Wake Lock API
reillyg: The implementation in Chromium always grants the permission to use a screen wake lock
… It seems reasonable to remove the permission requirement
… There was some accidental editorial issue, but there doesn't seem to be disagreement
<ghurlbot> Pull Request 326 Remove permission check (marcoscaceres)
RESOLUTION: Clean up and merge PR #326 to remove permission integration from Screen Wake Lock.
reillyg: The PR added a note that user agents may still deny
… Is there a standards position for screen wake lock? I think there's only a mailing list thread
Marcosc: Mozilla says "worth prototyping"
reillyg: Give the user agent information, and the user agent can then decide
ACTION: File WebKit standards position request for Screen Wake Lock API.
Marcosc: I file it now
System Wake Lock
reillyg: There wasn't much progress on this, but I made some with an explainer
… In the trend of APIs hinting declaratively to the user agent that they want to do something, if you play a video, this keeps the system awake and keeps the screen on, if you play audio, it keeps the system on, but not the screen.
… One of these requirements would be to tell the user agents that computation is going on
… This might help with service worker lifetime issues.
… If you want to keep the service worker alive for longer
tomayac: Isn't shared worker the tool for this
reillyg: Sometimes, but I want to convince folks that service worker is the response
tomayac: We talked once about background geolocation with system wake lock, would this be included?
reillyg: It could. The progress notification API (part of the upcoming explainer) could tell the service worker
… I could imagine if you created a progress notification for a run for example, this could work
… Every native platform seems to go in this direction. There's always some kind of notification required.
Marcosc: On iOS, it's modeled as "while using", "once", "always" for geolocation
… For geolocation
reillyg: There is an entitlement for this
… My mail app has an option to not use notifications, but to use IMAP
… You really can dismiss the permanent notification. Users can remove it manually if they really want to
reillyg: Android historically had a lot of background powers that it granted to apps
reillyg: This was tightened up. The web model could be something in between.
Marcosc: I have Lyft, and if I turn off the permission, it asks me if I want to use it.
reillyg: One of the non-geolocation use cases is processing and uploading photos.
… If you are a photo app, it will take, say 15s.
… The model we've been using on the web is to force the user to keep the device on
… I want to focus more on these use cases than on geolocation
… I finish something like processing a file or so
Theodore_Olsauskas-Warren: Why is geolocation not a use case that's interesting?
reillyg: I want to not open this can of worms
Marcosc: Background fetch might be able to deal with some of these use cases
reillyg: We have a tool that uses WebUSB that loads new images to a device
… If you close the tab, you don't want this to stop
… We can't make a list of all the use cases for example on USB
Marcosc: I like the idea of a signal
reillyg: Do you want a screen wake lock signal on different things was an idea initially
… The key thing is that a user agent cannot always tell what the user wants