14:58:06 RRSAgent has joined #webmachinelearning 14:58:10 logging to https://www.w3.org/2025/05/08-webmachinelearning-irc 14:58:10 inviting RRSAgent 14:58:10 RRSAgent, make logs Public 14:58:11 please title this meeting ("meeting: ..."), anssik 14:58:11 Meeting: WebML WG Teleconference – 8 May 2025 14:58:17 Chair: Anssi 14:58:21 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2025-05-08-wg-agenda.md 14:58:26 Scribe: Anssi 14:58:31 scribeNick: anssik 14:59:48 ningxin has joined #webmachinelearning 14:59:54 Winston has joined #webmachinelearning 14:59:58 gb, this is webmachinelearning/webnn 14:59:58 anssik, OK. 15:00:03 Present+ Anssi_Kostiainen 15:00:19 Regrets+ Reilly_Grant 15:00:24 Regrets+ Joshua_Bell 15:00:28 Regrets+ Christian_Liebel 15:00:34 Regrets+ Tarek_Ziade 15:01:04 zkis has joined #webmachinelearning 15:01:24 Present+ Dwayne_Robinson 15:01:33 Present+ Etienne_Noel 15:01:55 Mike_Wyrzykowski has joined #webmachinelearning 15:01:59 Present+ Mike_Wyrzykowski 15:02:06 Present+ Ningxin_Hu 15:02:10 RafaelCintron has joined #webmachinelearning 15:02:21 Present+ Rafael_Cintron 15:02:29 Present+ Winston_Chen 15:02:38 Present+ Zoltan_Kis 15:02:44 RRSAgent, draft minutes 15:02:45 I have made the request to generate https://www.w3.org/2025/05/08-webmachinelearning-minutes.html anssik 15:03:39 dwayner has joined #webmachinelearning 15:03:39 anssik: please welcome new participants! 15:03:46 ... Taylore Givens from Microsoft joining the WebML WG 15:04:33 Rafael: Taylore is working on Edge 15:05:11 anssik: Ehsan Toreini joining the WebML WG and CG affiliated with Samsung, focusing on privacy and trustworthiness of ML 15:05:25 ... and Guy Bary joining the WebML CG as an individual contributor 15:05:55 Topic: Incubations 15:07:40 Etienne: there's more discussion in the European meetings, so maybe if we have a few topics we can discuss them in the WG meeting 15:08:02 anssik: our previous Community Group meeting was an EU-APAC friendly one on Mon 28 April 15:08:11 anssik: I'll share a few key takeaways from that meeting, if you're interested in details please check the agenda and minutes for references: 15:08:14 -> https://github.com/webmachinelearning/meetings/blob/main/telcons/2025-04-28-cg-agenda.md 15:08:19 -> https://github.com/webmachinelearning/meetings/blob/main/telcons/2025-04-28-cg-minutes.md 15:08:44 anssik: the Community Group discussed a new proposal from Tarek/Mozilla called "Local inference web extension", Tarek opened a new issue for gathering input in our community proposals repo (thanks!) 15:08:49 -> Local inference web extension (proposal) https://github.com/webmachinelearning/proposals/issues/9 15:08:50 https://github.com/webmachinelearning/proposals/issues/9 -> Issue 9 Local Inference Web extension (by tarekziade) 15:09:34 anssik: we also discussed the following Prompt API feature requests: 15:09:56 ... - an output language support detection option -- this feature itself garnered support and contributions are welcome on naming suggestions 15:10:39 ... - multi-modal real-time capabilities -- we agreed this is potential future work, we also discussed the Web Speech API intersection with built-in AI APIs 15:11:34 ... - Model Context Protocol support -- a hot topic in the industry, we agreed this is an early exploration topic in the web context for interest folks, Christian has been doing MCP web exploration and I encouraged him to share with the community any progress 15:12:20 ... we acknowledged we received early wide review feedback for incubations, from the TAG for privacy and security and i18n for Language Detector, we're in fact ahead of expectations what comes to incubations review given we're already engaging with experts from the horizontal groups 15:12:25 ... any questions? 15:12:54 Topic: Operator specific issues 15:13:16 anssik: today we'll again focus our review and discussion on operator specific issues that reduce code complexity and improve maintainability 15:13:19 -> [operator specific] issues https://github.com/webmachinelearning/webnn/labels/operator%20specific 15:13:44 Subtopic: layerNormalization 15:13:49 anssik: issue #748 15:13:50 https://github.com/webmachinelearning/webnn/issues/748 -> Issue 748 Is there a reason to support out of order `axes` for layerNormalization? (by philloooo) [question] [operator specific] 15:14:31 ... Phillis reported findings from Core ML backend, when layerNormalization axes are out of order the result is wrong, sometimes 15:14:38 -> WebNN spec > the layerNormalization algorithm https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-layernormalization 15:15:08 anssik: Phillis asks why do we support unordered (aka inconsecutive) axes, are there any use cases? 15:15:20 ... it is noted we could emulate inconsecutive axes with transpose 15:15:40 ... Lisa also noted recently that ORT supports only last consecutive axes (initially discussed in issue #832) 15:15:41 https://github.com/webmachinelearning/webnn/issues/832 -> CLOSED Issue 832 Consider only support consecutive axes for LayerNormalization (by lisa0314) 15:16:08 ... a proposed fix would be to remove support for inconsecutive axes, any concerns with that? 15:16:44 Dwayne: the way I read, this mean inverted order, OK to have an axes 0 to 3 and skip over 1, not 1 where axes are in decreasing order 15:16:49 ... I was looking at the test case 15:17:40 Dwayne: I'll confirm whether this is non-increasing or non-adjecent, I'd be in inclined to keep non-adjecent 15:18:27 anssik: this issue was first identified by Austin via w-p-t test failures while running test on macOS, see Chromium CL comments: 15:18:27 15:18:27 -> https://chromium-review.googlesource.com/c/chromium/src/+/5516331/4/third_party/blink/web_tests/platform/mac/virtual/webnn-service-with-gpu/external/wpt/webnn/conformance_tests/layer_normalization.https.any_gpu-expected.txt#3 15:19:07 anssik: currently wpt.fyi tests do not run on macOS, latest passing results: 15:19:14 ... - PASS using CPU device on Chrome/Linux and Edge/Windows 15:19:14 ... - PASS using GPU device on Chrome/Linux 15:19:14 ... - PASS using NPU device on Chrome/Linux 15:20:09 Ningxin: wpt.fyi does not run against macOS Chrome, Chrome infrastructure does, run for every CL, Chrome folks are more familiar with the CI system 15:20:19 ... to get CLs merged need to pass on all platforms 15:21:08 Dwayne: I'll add a comment to clarify expectations 15:21:32 Ningxin: in Core ML docs there's no in order axes requirement, is this missing doc or implementation bug? 15:21:44 MikeW: I'll check if docs are missing or if this is unspecified 15:22:29 Subtopic: triangular 15:22:34 anssik: issue #768 15:22:35 https://github.com/webmachinelearning/webnn/issues/768 -> Issue 768 Consider removing or redesigning the `triangular` operator (by a-sully) [operator specific] 15:22:45 -> WebNN spec > triangular https://www.w3.org/TR/webnn/#api-mlgraphbuilder-triangular 15:23:08 anssik: Austin initially reported on Oct 2024 that all backends (DML, TFLite, Core ML) emulate triagular with a mask 15:23:19 ... Austin proposed to consider removing triangular 15:23:48 ... triangular was added as part of initial ops for transformers support in 2023 15:24:03 ... Dwayne shared a very detailed investigation in the issue (much thanks!) 15:24:24 Joshua_Lochner has joined #webmachinelearning 15:24:24 ... and explained the specific triangular behaviour was inspired by numpy.tri*, torch.tri*, onnx.ai.Trilu, tf.linalg.diag APIs 15:24:40 ... and how the Core ML's band_part missing the shift offset parameter causes an incompatibility 15:24:47 Apologies for the late join! Had to take dogs to the vet. 15:24:48 Present+ Joshua_Lochner 15:25:05 ... and shared a use case: Stable Diffusion's text encoder uses triangular 15:25:13 ... but noted only a few occurrences within a single model 15:25:36 ... Dwayne seems to also suggest the removal of triangular is justified 15:26:22 Dwayne: there's new information on Trilu ops popularity 15:26:24 I can get that information with https://github.com/huggingface/transformers.js-benchmarking. Give me a second 15:26:31 -> https://github.com/webmachinelearning/webnn/issues/768#issuecomment-2863362879 15:26:31 https://github.com/webmachinelearning/webnn/issues/768 -> Issue 768 Consider removing or redesigning the `triangular` operator (by a-sully) [operator specific] 15:26:43 Dwayne: there's decomp we could use 15:27:46 https://huggingface.co/datasets/onnx-community/model-explorer 15:27:58 Joshua: I published the full dataset ^ 15:28:30 ... this gives a good idea what's used in the wild 15:28:47 Dwayne: knowing the number of occurrences within a model is also useful 15:28:59 Joshua: Trilu op was of interest, I can check that soon 15:29:27 Dwayne: before removing triangular would like to check with new information at hand 15:29:52 q? 15:30:13 Ningxin: do we know any native optimized implementations for Trilu? 15:30:39 ... that'd help optimize the kernels and improve performance of models using this op, I shared OpenVINO implementation, but it's also decomposed 15:30:52 Dwayne: DML one does 15:30:59 Ningxin: please share the DML details as a comment 15:31:01 Dwayne: will do 15:31:02 q? 15:31:22 Subtopic: sign 15:31:27 anssik: issue #845 15:31:27 https://github.com/webmachinelearning/webnn/issues/845 -> Issue 845 The allowed data types of input operand for `sign` operator should also include `int64` type (by BruceDai) [operator specific] 15:31:56 ... while updating the wpt tests for the sign operator, Bruce noticed the sign does not support int64 data type 15:32:03 ... int64 data type was in Dwayne's original transformer ops proposal (from 2024) for sign: 15:32:07 -> https://github.com/webmachinelearning/webnn/issues/375#issuecomment-2292466613 15:32:07 https://github.com/webmachinelearning/webnn/issues/375 -> CLOSED Issue 375 Support for transformers (by dontcallmedom) [opset] 15:32:20 anssik: the reason for removing int64 was the lack of Core ML support 15:32:38 ... however, since then the WebNN API added opSupportLimits API that allows checking for supported data types 15:32:54 ... this seems analogous to how WebGPU allows developers to test for various GPU texture format support: 15:33:04 -> WebGPU spec > GPUTextureFormat https://www.w3.org/TR/webgpu/#enumdef-gputextureformat 15:33:53 https://chromium-review.googlesource.com/c/chromium/src/+/6489214 15:33:58 anssik: any concerns with adding int64 data type support considering opSupportLimits API gives an option for developers to detect support? 15:34:18 Dwayne: I see test cases for int64 are already in wpt 15:34:19 q+ 15:34:30 ack Mike_Wyrzykowski 15:34:39 MikeW: is this support an optional feature? 15:34:42 Dwayne: correct 15:34:48 MikeW: that'd be fine with me 15:35:18 anssik: it seems we have consensus, editors are free to submit a spec PR 15:35:19 q? 15:35:34 q? 15:35:43 Topic: WebNN wide review 15:36:21 anssik: I wanted to have an interim discussion on the wide review feedback we've received so we stay on top of the latest, and can agree on our response and react swiftly 15:36:25 ... the wide review tracker has been updated: 15:36:30 ... the wide review tracker has been updated: 15:36:35 -> https://github.com/webmachinelearning/webnn/issues/239#issuecomment-2740740891 15:36:35 https://github.com/webmachinelearning/webnn/issues/239 -> Issue 239 Wide review tracker (by anssiko) [process] 15:36:41 s/... the wide review tracker has been updated:// 15:36:49 Subtopic: Accessibility 15:37:00 anssik: for Accessibility review, we have draft feedback 15:37:05 -> https://github.com/w3c/apa/issues/350 15:37:06 https://github.com/w3c/apa/issues/350 -> Issue 350 WebNN Review (APA thread) (by matatk) 15:37:41 anssik: in summary, the feedback suggests a11y considerations for use cases 15:38:11 ... as a response to this feedback, I’d propose we add a note similar to what we have for privacy at the top of use case section 15:38:11 15:38:11 -> https://www.w3.org/TR/webnn/#usecases 15:38:41 anssik: this feedback does not directly propose changes to the WebNN API surface, but suggests e.g. image and video elements that are part of the web-based AI experience are accessible 15:38:47 anssik: this proposed a11y informed text would be non-normative and worded e.g. “authors are encouraged to …” to avoid any normative language (MUST, SHOULD) 15:38:57 anssik: when we get the official a11y review feedback, I will prepare a group's response for your review 15:39:04 ... any comments? 15:39:37 Subtopic: Architecture/TAG 15:39:46 anssik: for Architecture review by the TAG, our request is triaged with a focus on API design 15:39:51 ... no feedback received yet 15:40:02 Subtopic: Internationalisation 15:40:08 anssik: for Internationalisation review, we received a suggestion recorded in a new issue: 15:40:11 -> https://github.com/webmachinelearning/webnn/issues/837 15:40:11 https://github.com/webmachinelearning/webnn/issues/837 -> Issue 837 String metadata and localization for operator labels (by xfq) [i18n-needs-resolution] 15:40:20 anssik: the review feedback is about string metadata and localization for operator labels 15:40:25 anssik: Josh crafted a PR with a proposed response (thanks!): 15:40:30 -> https://github.com/webmachinelearning/webnn/pull/841 15:40:30 https://github.com/webmachinelearning/webnn/pull/841 -> Pull Request 841 Add notes regarding label usage, provided by i18n review (by inexorabletash) 15:40:38 anssik: the proposal is to add informative notes notes regarding label usage per i18n reviewers suggestions 15:40:42 ... I'm expecting us to merge this PR after the remaining minor comments are addressed 15:40:49 ... any questions or comments? 15:41:11 Subtopic: Privacy 15:41:36 anssik: for privacy review, we received one question: "does [opSupportLimits] only provide information about what the browser provides? When you say "roughly corresponds" what do you mean?" 15:41:40 -> https://github.com/w3cping/privacy-request/issues/156#issuecomment-2813166867 15:41:41 https://github.com/w3cping/privacy-request/issues/156 -> Issue 156 Web Neural Network API 2025-03-20 > 2025-06-20 (by anssiko) [CR] [pending] [REVIEW REQUESTED] 15:42:01 ... I responded explaining the WebNN API does not directly provide any information about the underlying ML APIs or libraries 15:42:36 ... I also noted the feature support per operator can be also inferred from the OS and browser version 15:42:48 ... the WebNN API is on the Privacy WG's 2025-05-15 (next Friday) agenda so I expect to get full review feedback soon 15:42:50 q? 15:43:16 Subtopic: Security 15:43:23 anssik: for Security review, no feedback yet 15:43:39 anssik: that's the latest on wide review, questions, comments? 15:44:58 Topic: Explainer updates 15:45:19 Subtopic: WebNN explainer 15:45:23 anssik: the WebNN API explainer gives a high-level overview of the API 15:45:28 -> WebNN explainer https://github.com/webmachinelearning/webnn/blob/main/explainer.md 15:46:02 anssik: we received some detailed feedback from an indivual outside the group (thanks!) and suggestion for explainer updates 15:46:19 ... this feedback is particularly helpful because it comes from a person who is not very familiar with this API, so we can expect also other early adopters to have similar questions 15:46:40 ... feedback in issue #840 15:46:41 https://github.com/webmachinelearning/webnn/issues/840 -> Issue 840 What are the options when some Native NN API isn't available? (by iampi31415) [question] 15:47:06 ... I think it is helpful to address these in an explainer update 15:47:17 anssik: possible WebNN explainer updates: 15:48:01 ... - revise the diagram, remove Android NN API that is deprecated, replace ONNX.js with ORT Web 15:48:23 ... - ML accelerators -> ML accelerators aka NPUs, TPUs, XPUs 15:48:39 ... - expand Considered alternatives explaining the complementary nature of WebNN and WebGPU 15:49:07 ... e.g. along the lines of Reilly's response in the issue: 15:49:20 ... WebGPU shaders can be highly-optimized for a specific model 15:49:25 ... OTOH WebNN ops are highly-optimized for the latest hardware generation 15:49:41 ... also worth noting these two APIs can now work better together thanks to MLTensor and related improvements 15:50:33 Subtopic: MLTensor 15:50:43 -> MLTensor explainer https://github.com/webmachinelearning/webnn/blob/main/mltensor-explainer.md 15:50:54 anssik: Bryan submitted a PR #844 to reflect the current implementation (WIP) 15:50:55 https://github.com/webmachinelearning/webnn/pull/844 -> Pull Request 844 MLTensor explainer: replace "import buffer" with "export tensor" (by bbernhar) 15:51:00 ... for explainer, the change is basically replacing "import buffer" with "export tensor" 15:51:05 ... with two reviews this PR is good to merge 15:51:24 ... PR #844 15:51:50 Dwayne: I'll look at this today and merge 15:52:08 ... there will be a follow-up PR for corresponding spec changes 15:52:25 Topic: Query supported devices 15:52:39 anssik: we've discussed this feature carefully and I see there are diverse use cases for querying supported devices 15:52:52 ... we split this issue in two, "before graph compilation" and "after graph compilation" 15:53:04 ... before going to those issues, I'd like to discuss what I consider the generic things we seem to be getting an agreement on 15:53:14 ... I think we agree we want to document the motivating use cases in the explainer, so we're use case driven 15:53:19 -> Device Selection explainer https://github.com/webmachinelearning/webnn/blob/main/device-selection-explainer.md 15:54:01 anssik: we've discussed preferences such as "prefer CPU", "prefer NPU", "prefer GPU", "maximum performance", "maximum efficiency", "minimum overall power" that could be mapped to and explained through use cases 15:54:30 ... and only then mapped to implementations to assess implementation feasibility and future proofness 15:54:41 ... considering these preferences would be merely hints, they should be implementable on any backend 15:55:36 Zoltan: we should discuss on the issue and distill consensus on the explainer, I'm cautious on "prefer NPU" 15:56:18 ... we should go by use cases, prefer issue discussion 15:56:19 q? 15:56:49 q+ 15:56:58 anssik: we discussed hard-coding some device names such as "npu" may not make sense and it was clear we want to use more future-proof names 15:57:29 ... "cpu" -> "where JS and Wasm execute", "gpu" -> "where WebGL and WebGPU programs execute", "other" -> "!cpu && !gpu" 15:57:30 q? 15:57:33 ack Mike_Wyrzykowski 15:57:50 q+ 15:57:58 MikeW: I was going to say I agree with what Zoltan said, prefer use case driven spec feature development and issue discussion 15:58:00 ack RafaelCintron 15:58:43 RafaelCintron: I just want to say, it is important to frame these as hints and requests, let implementations to do the ultimate decision, you may not always get what you want, but what is best for you in the end 15:58:45 q? 15:59:46 Subtopic: Query supported devices after graph compilation 15:59:55 anssik: issue #836 15:59:55 https://github.com/webmachinelearning/webnn/issues/836 -> Issue 836 Get devices used for a graph after graph compilation (by philloooo) [device selection] 16:00:03 ... this newly opened issue considers "after graph compilation" situation 16:00:09 ... Phillis explains: 16:00:23 ... "Eventually which devices are used can only be known after a model is provided. It's a different problem from the context level device query mechanism [before graph compilation]." 16:00:42 ... from implementation point of view, Core ML's MLComputePlan gives op level device selections after a model is compiled before calling dispatch 16:00:48 -> Core ML > MLComputePlan https://developer.apple.com/documentation/coreml/mlcomputeplan-85vdw 16:01:06 anssik: also TFLite shares information on which delegate gets used for each op after the graph is compiled 16:01:29 ... the proposal for WebNN suggests the following information could be attached to MLGraph that represents a compiled graph: 16:01:45 ... - devices (plural) that will be used to execute the graph 16:01:53 ... the examples in the issue illustrate this the best (consider devices names placeholders): 16:01:59 ``` 16:01:59 const graph = await builder.build({'C': C}); 16:01:59 console.log(graph.devices) // ['cpu', 'npu'] 16:01:59 16:01:59 const graph = await builder.build({'C': C}); 16:02:00 console.log(graph.devices) // ['cpu', 'npu'] 16:02:00 16:02:00 console.log(graph.deviceForOperations) 16:02:01 16:02:01 { 16:02:01 "add_1": "cpu", 16:02:02 "conv2d_2": "npu" 16:02:02 } 16:02:02 ``` 16:02:29 anssik: open questions: 16:02:39 ... - how to generate identifiers for each op: automatically generate or use label and return only ops with labels attached to them? 16:02:51 ... - is graph.devices enough, or does op level device selection have strong use cases? 16:03:18 ... in general, I'd like to hear from the group if we should made progress with both "before" and "after" issues together, or should we work in phases one before another and in which order? 16:03:19 q? 16:03:29 q? 16:03:56 Zoltan: I'm interested in how we'd use the information we get for op-level 16:04:23 anssik: looking for use cases for this op-level granularity as well 16:04:24 q? 16:04:54 RRSAgent, draft minutes 16:04:55 I have made the request to generate https://www.w3.org/2025/05/08-webmachinelearning-minutes.html anssik 16:05:42 s/Subtopic: MLTensor/Subtopic: MLTensor explainer 16:24:23 s/and requests/and preferences 16:24:52 s/let implementations to/let implementations 16:25:46 s/made progress/make progress 16:26:08 RRSAgent, draft minutes 16:26:09 I have made the request to generate https://www.w3.org/2025/05/08-webmachinelearning-minutes.html anssik 18:31:52 Zakim has left #webmachinelearning