Meeting minutes
Announcements
anssik: Model Loader API Dev Trial in Chrome OS M102
… from WebML CG
… credits to Honglin & team
… See also documenting implementation status discussion
#223
<ghurlbot> Issue 223 Documenting implementation status (anssiko)
"Now this API is ready for dev-trial in M102.
Please notice that the API is currently chromeOS only.
It is available after turning on two flags:
"#enable-experimental-web-platform-features" and "#enable-machine-learning-model-loader-web-platform-api"
A simple demo is https://
CommandBuffer usage clarification
anssik: Discuss on how to address the WebGPU resource usage and sharing issues in preparation for the WebGPU WG review.
#264
<ghurlbot> Issue 264 CommandBuffer usage clarification: internal, external, both? (bbernhar)
anssik: as a reminder, we are expected to reach out to WebGPU to review our design once settled or to seek guidance if we have multiple alternatives
gpuweb/gpuweb#2500
<ghurlbot> Issue 2500 Investigation: how WebNN / WebGPU interop could be happening (huningxin)
anssik: Bryan suggests in #264 that an open issue is around resource states that need to be correctly synchronized between WebNN and WebGPU or they will stomp over each other.
RafaelCintron: for this issue, we need to work with Bryan, I see his perspective, if the two impls are separate and away from each other, then need to agree if one system starts use WebGPU textures and the other does the same, they need to coordinate and this needs to happen on the impl level
… all comes down to someone try it and see what issues come up
Chai: my opinion is that I don't think this affects the API, I think it is important to agree how the impl should behave in terms of the resource that flows between the two APIs
… we could have a note section in the spec to describe interaction and resource sharing rules for implementers
… there could be a description how the resource could be shared, not affecting the design of the API
… resource produced by one API, then consumed by another, need to obey to the rules, but not affects the API per se
… I agree with Bryan we need to get in touch and understand how the implementation would go about this, similarly to any interop situation, you can define the API to work together, but if the impl disagrees you can't
anssik: are we ready to craft such a note for WebNN API to inform this interop?
Chai: not sure if there are other reps besides Bryan we could discuss with
… we can have informal connection with WebGPU friends on this matter
anssik: any other WebGPU-WebNN issues besides #2500?
<ghurlbot> Issue 2500 [not found]
RafaelCintron: not that I'm aware of
<ningxin_hu> i am good with current plan
WebGL interoperability requirements
anssik: Per discussion on our last call I started an RfC (or CfC) to drop WebGL interoperability requirements from the WebNN API spec
#268
<ghurlbot> Issue 268 RfC: drop WebGL interoperability requirements (anssiko) cr
anssik: the proposal to drop received +1 from Ningxin on our call
… also +1 from Chai in the issue
… no objections
… that issue discusses the rationale and concrete normative dependencies
… given adequate time has passed, I'll resolve to drop those requirements
<ghurlbot> Issue 263 [closed] Support asynchronous graph compilation (wchao1115)
RESOLUTION: Drop WebGL interoperability requirements from the WebNN API spec per #263
Privacy Review refresh
anssik: we have made strong progress with wide review thanks to your contributions
… the remaining review to close on is privacy review refresh
… "refresh" because we completed the initial privacy review in 2020/2021
Initial Privacy review recap
issue #119: [2020-11-05] Initial Self-Review Questionnaire response: Security and Privacy
PR #132: [2021-02-18] Add responses to the Self-Review Questionnaire: Security and Privacy
PR #170: [2021-05-27] Add initial Security and Privacy Considerations sections
Changes since initial privacy review
anssik: Review changes since initial privacy review 2021-05-27 for any privacy-impacting changes:
PR #259: [2022-05-21] Add privacy considerations for cached/persisted data
HTML diff for changes since 2021-05-27
All PRs merged since 2021-05-27
anssik: It may be easier to review merged PRs for privacy impact
Open privacy-tracker issues
issue #175: [2021-06-03] Related WebGPU/GL Security and Privacy Considerations
anssik: #175 is a great opportunity to folks interested in WebGPU to review its Privacy Considerations given our interop expectation
anssik: a path forward would be to cross-reference specific applicable WebGPU Privacy Considerations from the WebNN API spec
… currently we say in WebNN Privacy Considerations:
"In general, implementers of this API are expected to be familiar with the WebGPU Privacy Considerations.
anssik: is there any WebGPU privacy focused person?
RafaelCintron: I think it is a group effort
… a lot of written these is informed by past WebGL experience, e.g. timestamp changes in WebGL that were carried over
<jonathan_> apologies, have to go to another meeting now
RafaelCintron: many consideration will be same as for WebGPU, e.g. validate all inputs
… web developer cannot give bad data, or index out of bounds, use buffers that do not belong to them
… timing attacks section, we don't have an API that can be used to time timings other than performance.now()
… driver bugs asks us to have conformance tests, we have similar issues with WebNN
… we don't have write your own shader worries
anssik: issue #169: [2021-05-13] Device selection with MLDevicePreference and MLPowerPreference
<ghurlbot> Issue 169 Device selection with MLDevicePreference and MLPowerPreference (anssiko) privacy-tracker
anssik: in #169 we're discussing whether MLDevicePreference and MLPowerPreference are hints (implementation to decide) or normative (implementation fails if cannot satisfy)
… if normative, has privacy implications as web content can learn some hardware capabilities
… current level of abstraction for devices ("cpu", "gpu") and power ("default", "high-performance", "low-power") do not offer very unique information for fingerprinting
… for Privacy Review refresh purposes it'd be good to say whether we consider MLDevicePreference and MLPowerPreference hints or normative?
Chai: the content in this issue is a bit stale, no MLDevicePreference, now we have a MLDeviceType
anssik: MLDeviceType is now normative, will fail if the impl cannot satisfy?
Chai: right
… the only hint is MLPowerPreference
anssik: everyone happy with this design?
[agreement]
issue #85: [2020-08-27] Fingerprinting via matmul webmachinelearning/webnn#85
<ghurlbot> Issue 85 Fingerprinting via matmul (kpu) privacy-tracker
<ghurlbot> Issue 85 Fingerprinting via matmul (kpu) privacy-tracker
anssik: per this feedback, we added the following to Privacy Considerations:
"An execution time analysis may reveal indirectly the performance of the underlying platform's neural network hardware acceleration capabilities relative to another underlying platform."
anssik: since then, we received two pieces of feedback from Google Chrome privacy reviewers:
… 1st comment:
anssik: "1) The section does a great job covering machine-specific performance considerations. Would it be possible to address the issue of "machine-specific artifacts" as addressed in the WebGPU document."
WebGPU Privacy Considerations: Machine-specific artifacts
Rafael: skimming this over, I think this also applies to WebNN, given compute units scheduling
<RafaelCintron> From WebGPU Spec: "Performance differences are relatively intractable, but also relatively low-signal (as with JS execution performance)."
anssik: Machine-specific artifacts should be explicit surfaced in the WebNN spec Privacy Considerations?
RafaelCintron: agreed
RESOLUTION: Surface WebGPU-inspired machine-specific artifacts in the WebNN spec Privacy Considerations
"I see that the section points to the WebGPU Privacy Considerations section. Are there any additional considerations or orthogonal fingerprinting vectors for WebNN, such as when dedicated ML hardware accelerators are in use?"
anssik: I think the Device Selection design that we just settled on is a response to this
… any other thoughts?
ningxin_hu: as well as MLDeviceType, there are ML accelerators, previous discussion around data types these ML accelerators support, e.g. they may only support float16 or int8
… I remember we discussed the behaviour if the data type is not supported, how the API should react to that
… possible fingerprinting vector, but now that ML accelerator support is not in the initial scope, not an immediate concern
<ghurlbot> Issue 85 Fingerprinting via matmul (kpu) privacy-tracker
<ghurlbot> Issue 169 Device selection with MLDevicePreference and MLPowerPreference (anssiko) privacy-tracker
RESOLUTION: Request Privacy Review refresh after the WebGPU (#175 and #85) and Device Selection (#169) privacy comments have been acknowledged in Privacy Considerations
issue #7: [2018-10-26] Cosine similarity between two facial features and cross origin tracking
anssik: it seems like there are no technical measures to mitigate this
… but this is considered as part of our Ethical Principles for Web Machine Learning effort that started after this issues was opened
Ethical Principles for Web Machine Learning
… I'm considering transferring this issue to that repo, any concerns?
<ningxin_hu> sgtm
Open issues and pull requests
"Ethical Content Filtering" use case
anssik: I've asked Humera to share what concrete suggestions she would like to see in the WebNN API to support ML-based content filtering implementation
#253
<ghurlbot> Pull Request 253 Add "Ethical Content Filtering" use case to WebNN specs (humeranoor)
Humera: idea is that when we look at WebNN use cases, we should have WebNN-based content filtering as one use case
… many ethical and fair use content blocking mechanism rely on ML
… the concept is that some things need to updated of included in the WebNN
… concrete features
… network level blocking, DOM-based inferencing
… for network blocking some features are in place, but at risk of being deprecated
… for DOM access, we need to look at the page on DOM level and make decisions on blocking some of its elements
https://