14:57:24 RRSAgent has joined #webmachinelearning 14:57:29 logging to https://www.w3.org/2024/12/19-webmachinelearning-irc 14:57:29 RRSAgent, make logs Public 14:57:30 please title this meeting ("meeting: ..."), anssik 14:57:30 Meeting: WebML WG Teleconference – 19 December 2024 14:57:35 Chair: Anssi 14:57:44 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-12-19-wg-agenda.md 14:57:44 Scribe: Anssi 14:57:49 scribeNick: anssik 14:57:58 Present+ Anssi_Kostiainen 14:58:04 RRSAgent, draft minutes 14:58:05 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 14:58:52 Present+ Zoltan_Kis 15:00:15 zkis has joined #webmachinelearning 15:00:16 ningxin has joined #webmachinelearning 15:00:29 Present+ Dwayne_Robinson 15:00:52 Present+ Christian_Liebel 15:01:00 Present+ Ningxin_Hu 15:01:28 Present+ Yuichiro_Tachibana 15:01:32 Present+ Dominique_Hazael-Massieux 15:01:36 christianliebel has joined #webmachinelearning 15:01:38 jsbell has joined #webmachinelearning 15:01:45 Present+ Joshua_Bell 15:01:50 present+ 15:02:14 McCool has joined #webmachinelearning 15:02:29 anssik: Welcome to our new participants Yang Gu, Jiajia Qin and Shaobo Yan from Microsoft 15:02:46 ... also warm welcome to Yuichiro Tachibana from Hugging Face joining as a guest! 15:04:05 Yuichiro: I met W3C members at a conference discussing Transformers.js, thanks for inviting me, interested in client-side ML execution 15:04:54 Topic: WebML Working Group Charter update 15:05:04 gb, this is w3c/machine-learning-charter 15:05:05 anssik, OK. 15:05:14 anssik: Working Group charter is up for renewal in 2025 15:05:38 ... today we'll review the current charter, discuss and solicit input on proposed changes, triage open charter issues, and confirm the timeline 15:05:49 anssik: the key principle for WG rechartering is "if it ain't broken don't change it" 15:05:56 ... this concerns the scope expansion specifically 15:06:08 ... if the scope expands, existing participants need to re-join 15:06:40 Subtopic: Current WG Charter 15:06:45 -> WG Charter 2023-2024 https://www.w3.org/2023/04/web-machine-learning-charter.html 15:07:22 anssik: Motivation and Background is informational 15:07:29 RafaelCintron has joined #webmachinelearning 15:07:56 ... Scope is an important section and is future-proofed with "well-known model architectures" and "major platform APIs" 15:08:12 DwayneR has joined #webmachinelearning 15:08:32 Yuichi has joined #webmachinelearning 15:08:35 ... Out of Scope section includes training, hardware features and hardware algorithms 15:09:03 ... Deliverables are WebNN API, Model Loader API as tentative, Ethical Principles WG Note 15:09:29 ... Success Criteria is the standard one, two independent interoperable implementations 15:10:00 ... Coordination enumerates groups we usually work with, but does not prevent us from working with other groups or projects outside this list 15:10:08 ... the rest is standard charter boilerplate 15:10:54 Dom: good overview Anssi, Scope and Deliverables are the most important 15:11:12 ... think no need to change Scope or add new Deliverables 15:11:38 ... Dom submitted a PR #39 to align with the latest charter template, thanks! 15:11:39 https://github.com/w3c/machine-learning-charter/pull/39 -> Pull Request 39 Align with latest charter template (by dontcallmedom) 15:11:45 -> Diff https://pr-preview.s3.amazonaws.com/w3c/machine-learning-charter/39/dd0dbc1...429a03a.html 15:12:01 q+ 15:12:03 ... any questions or comments about the current charter or Dom's proposed changes? 15:12:08 ack jsbell 15:12:24 anssik: next we'll look at open charter issues 15:12:27 -> WG Charter open issues https://github.com/w3c/machine-learning-charter/issues 15:12:37 Subtopic: Model Loader API, keep as tentative or remove from scope? 15:12:41 anssik: issue #38 15:12:42 https://github.com/w3c/machine-learning-charter/issues/38 -> Issue 38 Model Loader API, keep as tentative or remove from scope? (by anssiko) 15:12:53 ... Model Loader API incubation has been on a pause since Feb 2023 and its known implementation was removed from Chromium after experimentation phase 15:13:07 ... unless there's interest, I'd propose to remove this from the WG Charter and revive this work in the WebML CG as appropriate 15:13:52 q? 15:14:15 jsbell: not currently being persued, supportive of removing 15:14:44 Subtopic: Core operator set, scope and coordination 15:14:50 anssik: issue #37 15:14:51 https://github.com/w3c/machine-learning-charter/issues/37 -> Issue 37 Core operator set, scope and coordination (by anssiko) 15:15:10 ... the current WG charter scope section is abstract enough to not warrant a revision to allow work on core op set to happen 15:15:28 ... we could update the informative list of major platform APIs if there are changes: 15:15:35 "The APIs in scope of this group are not tied to any particular platform and are implementable on top of existing major platform APIs, such as Android Neural Networks API, Windows DirectML, and macOS/iOS Metal Performance Shaders and Basic Neural Network Subroutines." 15:15:54 ... this is an open ended list, not being on this list does not exclude any platform APIs 15:16:23 anssik: related, in Coordination we note StableHLO, while we also consider MLIR Linalg, PyTorch Prims IR, TOSA for our compositional fundamentals research 15:16:44 ... we probably don't want to enumerate all possible projects (MLIR, PyTorch, TOSA), so perhaps it is more balanced to remove the StableHLO reference? 15:16:55 q? 15:17:50 Dom: being open-ended and informative, the only thing is this can be useful as a reminder of communities who to seek wide review from 15:18:12 q+ 15:18:16 ... perhaps under that lens, consider removing 15:18:30 Christian: seems good to remove, that seems quite specific one 15:18:31 ack jsbell 15:18:52 jsbell: we normally list WHATWG in charters, most specs have dependencies to WebIDL, Infra and other specs an standards 15:19:02 Dom: we usually do that when there's a specific requirement 15:19:29 ... e.g. new types for WebIDL required, any browser WG has a WHATWG relationship, would not put that as a hard requirement, can put it in if wanted 15:19:31 q? 15:20:06 q? 15:20:17 Subtopic: Task-based APIs and Prompt API 15:20:20 anssik: issue #36 15:20:21 https://github.com/w3c/machine-learning-charter/issues/36 -> Issue 36 Task-based APIs and Prompt API (by anssiko) 15:20:24 ... Task-based APIs and Prompt API were adopted into the WebML Community Group earlier this month 15:20:29 ... these APIs are now incubated in the WebML CG 15:20:46 ... proposal is to check for readiness for WebML WG adoption from time to time considering implementation experience, end user feedback 15:21:29 Dom: we could list them as tentative if we feel they are likely target for adoption, concern is this space is very active with interesting IP questions 15:21:48 ... bringing into charter might create friction in the AC review and require everyone re-join 15:22:07 ... proposal is to wait and see the traction 15:22:08 q? 15:22:37 q+ 15:22:46 Christian: personally, would support adding the APIs, but understand and appreciate Dom's perspective 15:23:12 ... compared to Model Loader API these APIs seem compatible 15:23:58 Dom: we are free to charter again 6 months from now when the incubations have made more progress 15:24:26 ... Model Loader API has different IPR scope from task-based APIs 15:24:49 q+ 15:24:55 q- 15:25:11 jsbell: given what Dom said, it has a very specific API called out, maybe in 6 months from now we would be in a better position to see what would make sense to capture as Tentative then 15:25:48 q? 15:26:22 Dom: as part of our communication to AC, I will make sure we mention these APIs have been adopted in the CG and are consideration for the WG future revision 15:26:36 q? 15:26:39 ack christianliebel 15:27:22 christianliebel: what Dom says makes sense, having publicly visible commitment we are looking at these new APIs, it makes sense to now add to the WG charter right now to not create friction 15:27:23 q? 15:27:32 Subtopic: Speech Synthesis 15:27:36 anssik: issue #31 15:27:37 https://github.com/w3c/machine-learning-charter/issues/31 -> Issue 31 Speech synthesis and machine learning (by r12a) [deferred] 15:27:48 ... suggestion to mention Speech Synthesis for symmetry with Speech Recognition in informative Motivation and Background 15:27:52 ... I propose we update the informative text e.g. as follows: 15:28:03 "Speech Recognition and Speech Synthesis enable computers to recognize and translate spoken language into text and vice versa." 15:28:08 q? 15:28:17 Subtopic: On-device training 15:28:27 anssik: no feedback that'd suggest we should bring on-device training in scope for the WG 15:28:47 ... issue #27 15:28:47 https://github.com/w3c/machine-learning-charter/issues/27 -> Issue 27 On-device training (by anssiko) [deferred] 15:29:14 Subtopic: Charter development timeline 15:29:22 ... I'll prepare an updated charter with Dom early new year for your review 15:29:51 ... then complete horizontal review, around mid-Feb we will initiate the AC review 15:29:57 ... new charter start date would be 2025-05-01 15:30:34 q? 15:31:06 Topic: Device selection abstractions update 15:31:09 gb, this is webmachinelearning/webnn 15:31:09 anssik, OK. 15:31:15 anssik: PR #784 15:31:16 https://github.com/webmachinelearning/webnn/pull/784 -> Pull Request 784 Add device selection explainer (by zolkis) 15:31:44 anssik: Zoltan has updated the explainer proposal, added new use cases, known implementation limitations, added MVP solution to remove explicit deviceType to make contexts device agnostic, allow multiple devices per context 15:31:55 -> Updated explainer (preview) https://github.com/webmachinelearning/webnn/blob/f45266fb1223988894b0ccad7701c41aa753c5f1/device-selection-explainer.md 15:32:41 Zoltan: folks will need some time to digest the space, we have documented intro, history, key use cases and requirements, considered alternatives 15:32:54 ... recently added Minimum Viable Solution for your review 15:33:20 ... one pain point is we were tied to device per context, while some platforms can execute on multiple devices 15:33:42 ... also our device selection mechanism did not map well to platform APIs 15:33:52 ... proposal is to: 15:33:54 ... - Remove MLDeviceType as explicit context option. 15:34:20 ... - Update MLContext so that it becomes device agnostic, or default/generic context. Allow supporting multiple devices with one context. 15:34:45 ... - Add notes to implementations on how to map power preference to devices. 15:35:11 ... - Improve the device selection hints in context options and define their implementation mappings. 15:35:21 ... - Check if requesting a certain device type or combination of devices is still a use case. 15:36:15 ... please review and chime in the PR if this direction has issues, otherwise I will proceed with a spec PR per this design 15:36:50 q+ 15:37:03 ack jsbell 15:37:18 jsbell: explainer looks great, support getting the explainer merged 15:37:27 q+ 15:37:31 ... where to capture feedback to support the explainer, is there an issue? 15:37:59 ... Google team is on a vacation for the next few weeks 15:38:14 Zoltan: I can wait over the holiday for feedback 15:39:02 q? 15:39:49 ack RafaelCintron 15:40:03 RafaelCintron: thanks for putting this together, couple of questions about the explainer 15:40:16 ... at the end it says "allow supporting multiple devices in one context" 15:40:39 ... does "other devices" mean a new type? 15:41:00 Zoltan: this would be abstracted away, if defining a context as a combination of devices, should be also possible 15:41:12 Rafael: low-latency, what kind of device would be low latency? 15:41:15 +1 that I had the same question as Rafael at first, so clarifying the text would be great. 15:41:47 Zoltan: low latency, if you want to optimize e.g. LLM throughput tell that to the implementation and it's do the best to satisfy that, consider it a hint 15:42:50 ... just a suggestion in which direction to extend the context options, I found low-latency hint in OpenVINO, we need to validate the use cases and craft hints based on that 15:43:24 https://blog.openvino.ai/blog-posts/automatic-device-selection-and-configuration 15:43:53 Rafael: high-performance and low-power I know how to deal with, they exist on Windows, low-latency I'm not familiar with, CPU could be low-latency 15:44:27 Zoltan: I'd spec these as hints that if they cannot be fulfilled they're not errors 15:45:30 Zoltan: PR #784 would be the perfect place for feedback 15:45:31 https://github.com/webmachinelearning/webnn/pull/784 -> Pull Request 784 Add device selection explainer (by zolkis) 15:45:36 +1 to provide feedback on PR 15:46:22 q? 15:46:44 Topic: WebNN Operator Update Wave 3 15:47:09 anssik: Dwayne has a WIP PR for op set Wave 3, thanks Dwayne! 15:47:29 ... OK to push WIP PR to upstream repo and mark it as a Draft 15:47:39 ... this enables PR Preview and CI checkers that can be helpful for development 15:47:45 ... and folks can help contribute 15:48:39 Dwayne: Austin has a few question on uint4 on CoreML, does not block the spec PR 15:49:04 Topic: Core op set: MLIR Linalg findings revisited 15:49:15 anssik: we discussed Dwayne's extensive mapping table month ago, Linalg specifically 15:49:18 -> Mapping table https://onedrive.live.com/view.aspx?resid=EE82F5C6F06C7371%21345450&authkey=!AK8f-RDTleqlLXE 15:49:38 ... I wanted to bump this topic to get feedback on the 6 primitive ops proposed for inclusion into WebNN API informed by this reasearch, they are: 15:50:43 -> 1-D convolution with no channels https://mlir.llvm.org/docs/Dialects/Linalg/#linalgconv_1d-linalgconv1dop 15:50:46 -> 3-D convolution with no channels conv_3d https://mlir.llvm.org/docs/Dialects/Linalg/#linalgconv_3d-linalgconv3dop 15:50:49 -> Fill output with random numbers fill_rng_2d https://mlir.llvm.org/docs/Dialects/Linalg/#linalgfill_rng_2d-linalgfillrng2dop 15:50:53 -> Sum pooling pooling_nchw_sum https://mlir.llvm.org/docs/Dialects/Linalg/#linalgpooling_nchw_sum-linalgpoolingnchwsumop 15:50:56 -> Min pooling pooling_nhwc_min https://mlir.llvm.org/docs/Dialects/Linalg/#linalgpooling_nhwc_min-linalgpoolingnhwcminop 15:50:59 -> Round(x) elementwise round https://mlir.llvm.org/docs/Dialects/Linalg/#linalground-linalgroundop 15:51:27 Dwayne: this set 6 is implementable across backends 15:51:43 ... no direct mapping for all of them, most directly implementable 15:52:06 ... I can fill in the table with all the ops, take a subset of three backends Chromium uses and show how they map to these 15:53:03 jsbell: following the usual process, if implemetable across backends, use cases, sounds good 15:54:00 Topic: Get to know Task-specific APIs and Prompt API 15:54:04 -> Incubations landing page https://webmachinelearning.github.io/incubations/ 15:54:49 anssik: companion WebML Community Group now incubates selected task-specific APIs to enable reuse of the built-in models that are distributed as part of the browser or the underlying software platform 15:55:03 q+ 15:55:07 ... as an observation, it seems Prompt API as a more general purpose and flexible API has received the most feedback 15:55:20 ... on our last meeting we discussed how to versioning the models, and it was suggested as a topic to be discussed in this group, adapters and LoRAs, model management 15:55:20 q? 15:55:23 q+ 15:55:24 ack dom 15:55:54 Dom: open-ended question, one topic I'd expect to be broad up is questions on ethical considerations for ML 15:56:23 ... would a model that is used by these APIs be documented in a way developers can learn how they're trained, bias, other qualities, a la Model Cards style info 15:56:43 ... as we think bringing specific models as part of API surface in browsers, thinking about ethical aspects is important 15:56:58 q? 15:57:05 ack christianliebel 15:57:36 Christian: good questions, touch to answer, I have given dozen of presentation on these APIs, can confirm Prompt API has the most interest 15:57:36 q+ 15:57:48 ... restricting only for extensions is problem for developers 15:58:00 ... ethical part, how to make sure the answers are safe? 15:58:10 ... is there filtering, can I query? 15:58:21 ... overall I'm happy we have these APIs here, and look forward to see more 15:59:00 ... for specific issues, output constraining is important, our customers use function calling, multi-modal, Prompt API represents what LLMs were year ago 15:59:09 q+ 15:59:14 q? 15:59:16 ack jsbell 15:59:45 jsbell: +1 to what Christian said, we want to work through the Prompt API issues, also Dom thanks for ethical considerations 15:59:58 +1 that WebNN delegates that question rather than solving it :) 16:00:21 ... in WebNN API there's transparency, but it pushes complexity to developer on how "responsible" the model is 16:00:56 jsbell: this is not the first time we have ML-backed APIs in the browser, we have Web Speech API 16:01:01 q? 16:01:04 ack McCool 16:01:37 McCool: STT and TTS, use cases for Prompt API? 16:01:51 jsbell: we'll explore multimodal in 2025, current APIs are only text-to-text 16:01:52 q? 16:02:04 Topic: Happy Holidays! 16:02:18 anssik: Thank You everyone for your significant contributions during 2024! 16:02:25 ... our Working Group accomplished a lot this year 16:02:30 ... a few highlights: 16:02:53 ... WebNN API evolved driven by research into popular more advanced models, more diverse implementations 16:03:03 ... WebNN API Candidate Rec Snapshot milestone was met in 2Q 2024 16:03:16 ... the WG made in total ~100 spec publications and merged +180 PRs this year 16:03:39 ... many new active contributors join, Dwayne as a co-editor, the group grew and diversied further 16:03:58 ... we converged on new API abstractions for tensors, device selection, defined op set principles 16:04:13 ... we improved the spec quality significantly with expert advice 16:04:38 ... we organized our first F2F in Anaheim and it was a blast 16:04:55 ... we made strong progress on the implementations across 3 backends, XPUs, multiple OSes 16:05:11 ... we witnessed positive buzz in the tech industry around WebNN, made a few keynote appearances 16:05:26 ... a lot of exciting demos and samples were published, wpt test coverage improved 16:05:31 ... and much more! 16:05:39 ... we're entering an exciting phase of development in 2025 16:05:57 ... the WebNN API is expected to get in the hands of more developers for large-scale trials, and more 16:06:03 ... feedback from developers and users will help guide our priorities 16:06:20 ... Happy Holidays everyone -- please relax, disconnect, and recharge 16:06:29 ... see you on our next call 16 Jan 2025! 16:07:02 RRSAgent, draft minutes 16:07:03 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 16:11:17 s/… Dom submitted/anssik: Dom submitted 16:11:20 RRSAgent, draft minutes 16:11:21 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 16:16:48 s/to now add/to not add 16:20:21 s/broad up/brought up 16:21:00 s/touch/tough 16:22:28 s/contributors join/contributors joined 16:23:17 RRSAgent, draft minutes 16:23:19 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 16:24:06 s/Dom submitted a PR/submitted a PR 16:24:08 RRSAgent, draft minutes 16:24:09 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 16:27:30 s/diversied/diversified 16:27:32 RRSAgent, draft minutes 16:27:33 I have made the request to generate https://www.w3.org/2024/12/19-webmachinelearning-minutes.html anssik 17:46:39 zkis has joined #webmachinelearning 18:30:50 Zakim has left #webmachinelearning