14:49:44 <RRSAgent> RRSAgent has joined #webmachinelearning
14:49:48 <RRSAgent> logging to https://www.w3.org/2026/06/04-webmachinelearning-irc
14:49:48 <Zakim> RRSAgent, make logs Public
14:49:49 <Zakim> please title this meeting ("meeting: ..."), anssik
14:49:50 <anssik> Chair: Anssi
14:50:01 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2026-06-04-wg-agenda.md
14:50:01 <anssik> Scribe: Anssi
14:50:01 <anssik> scribeNick: anssik
14:50:03 <anssik> gb, this is webmachinelearning/webnn
14:50:03 <gb> anssik, OK.
14:50:08 <anssik> Present+ Anssi_Kostiainen
14:50:17 <anssik> Regrets+ Markus_Handell
14:50:17 <anssik> RRSAgent, draft minutes
14:50:19 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
14:50:53 <anssik> Meeting: WebML WG Teleconference – 4 June 2026
14:50:54 <anssik> RRSAgent, draft minutes
14:50:56 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
14:51:54 <mtavenrath> mtavenrath has joined #webmachinelearning
14:58:22 <DwayneR> DwayneR has joined #webmachinelearning
14:58:40 <anssik> Present+ Dwayne_Robinson
14:58:45 <anssik> Present+ Markus_Tavenrath
15:00:36 <Mike_Wyrzykowski> Mike_Wyrzykowski has joined #webmachinelearning
15:00:45 <anssik> Present+ Jonathan_Ding
15:00:54 <anssik> Present+ Mike_Wyrzykowski
15:01:12 <anssik> RRSAgent, draft minutes
15:01:14 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
15:01:21 <anssik> Present+ Ningxin_Hu
15:01:31 <anssik> Present+ Rafael_Cintron
15:01:36 <zolkis> zolkis has joined #webmachinelearning
15:01:39 <anssik> Present+ Zoltan_Kis
15:01:48 <anssik> RRSAgent, draft minutes
15:01:50 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
15:02:18 <anssik> Present+ Ehsan_Toreini
15:03:03 <anssik> Present+ Reilly_Grant
15:03:31 <anssik> Topic: Announcements
15:03:39 <anssik> Subtopic: TPAC 2026
15:03:40 <RafaelCintron> RafaelCintron has joined #webmachinelearning
15:03:53 <anssik> Anssi: TPAC 2026, an annual W3C conference, is in Dublin, Ireland on 26-30 October 2026
15:04:05 <anssik> ... during that week, the W3C groups gather to resolve challenging issues and discuss future directions for the web platform
15:04:13 <anssik> ... TPAC group meetings take place Mon, Tue, Thu and Fri
15:04:17 <anssik> ... Wed is for breakout sessions and social events
15:04:24 <anssik> ... my working assumption is this Working Group will meet during the TPAC week
15:04:24 <ningxin> ningxin has joined #webmachinelearning
15:04:34 <anssik> ... before I request a meeting time from the TPAC planners, I'd like to hear from the group about any potential conflicts with other groups or preferences for meeting times during TPAC week
15:04:39 <anssik> ... historically, we have met on Monday
15:04:44 <anssik> ... assuming also other groups tend to stick with their established schedules, that is my default preference
15:04:47 <Ehsan> Ehsan has joined #webmachinelearning
15:04:55 <anssik> ... but I'm open to other suggestions
15:04:59 <anssik> ... some of the group have shared their schedule preferences already:
15:05:02 <anssik> -> https://github.com/w3c/tpac2026-meetings/issues
15:05:11 <anssik> Anssi: TPAC 2026 website is not yet live, I'll share more information when it becomes available
15:05:19 <anssik> Anssi: questions, comments?
15:05:29 <anssik> Topic: Web Neural Network API
15:05:46 <anssik> Subtopic: Dynamic AI Offloading Protocol (DAOP) update
15:06:06 <anssik> Anssi: the Dynamic AI Offloading Protocol (DAOP) incubation is ongoing and I'm pleased to welcome Jonathan Ding to share an update on this explorative work
15:06:19 <anssik> ... learnings from this incubation will inform WebNN feature work wrt QoS estimation, dynamic execution routing and other features
15:06:24 <anssik> ... Jonathan presents first and we will have a discussion after
15:06:29 <anssik> ... we'll timebox this to 15 minutes combined
15:06:32 <anssik> ... relevant presentation materials will be shared after the meeting
15:06:36 <anssik> Anssi: Jonathan, please take it away
15:06:44 <anssik> Slideset: daop-slides
15:06:53 <anssik> [slide 2]
15:06:57 <anssik> Jonathan: DAOP Exploration for Hybrid LLM / Agent scenarios
15:07:08 <anssik> ... Dynamically offload to device / fallback to cloud
15:07:15 <anssik> ... Routing based on (latency, accuracy)
15:07:23 <anssik> ... - Latency = Estimate(Prompt, Model)
15:07:36 <anssik> ... - Accuracy = P(Correct | Prompt, Model)
15:08:05 <anssik> Present+ Sarah_Drasner
15:08:17 <anssik> Jonathan: - Others (cost, privacy, …) in future
15:08:53 <anssik> ... PoC WIP – pre-trained offline models for routing
15:09:18 <anssik> ... - Latency based on static cost models with microbenchmarks / profiling data on operators
15:09:31 <anssik> ... - Accuracy based on matrix factorization algorithms w/ selected LLM benchmarks – revised on RouteLLM
15:09:39 <anssik> ... Deployment Considerations
15:09:50 <anssik> ... - Proposing W3C standards in WebML WG – QoS, Events
15:09:53 <anssik> ... - Routing Layer on top of Built-in AI of browser / Web runtime
15:09:59 <anssik> [slide 3]
15:10:04 <anssik> Jonathan: Latency Estimation
15:10:10 <anssik> ... Estimation Model
15:10:15 <anssik> ... - Static Cost Model on Raw / Fused Compute Graph
15:10:48 <anssik> ... - Piecewise Log-space Interpolation in Parameter Space for key Ops, e.g.
15:11:05 <anssik> ... -- (M, K, N, dtype) for MUL_MAT
15:11:10 <anssik> ... -- (seqQ, seqKV, Q-heads, KV-heads) for FLASH_ATTN
15:11:16 <anssik> ... Data Sources
15:11:45 <anssik> ... - COLD:  One-time microbenchmarks (~90s, ~1000 pts)
15:11:54 <anssik> ... - WARM:  Profiling data from daily inference [future]
15:12:00 <anssik> [slide 4]
15:12:05 <anssik> Jonathan: Initial Experiments about Accuracy based offloading
15:12:16 <anssik> ... [ Jonathan presents the Two-Level Conservative Routing Decision Flow and performance of the PoC routing model ]
15:14:54 <zolkis> q+
15:15:07 <anssik> Anssi: thank you Jonathan!
15:15:10 <anssik> ack zolkis
15:15:40 <anssik> Zoltan: when you run local models, it is in one part, but harness matters so context and memory management to be  taken into account, or is it like running a model in the cloud?
15:16:36 <anssik> Jonathan: you raise a good point that not all text inside your prompt are important, e.g. with large context perhaps only beginning, the system prompt and the end are important
15:16:56 <anssik> ... this is important for the accuracy consideration, you need the rest for prefill, for accuracy need to do something smarter
15:16:57 <anssik> q?
15:17:11 <anssik> Zoltan: thanks for the response
15:19:28 <anssik> Jonathan: my next step would be add profiling to the inference engine, to get data from daily runs, for privacy considerations, we don't want to expose any such data directly
15:19:55 <anssik> ... for a new model you'd ask a questions what type of latency I should expect given a particular context length
15:20:19 <anssik> s/questions/question
15:20:26 <anssik> Jonathan: a
15:20:33 <anssik> s/Jonathan: a//
15:21:22 <anssik> Jonathan: latency estimate can be bucketed in a way it can provide useful information for the web developer to inform model offload decision while preserving privacy
15:22:49 <anssik> Jonathan: for WebNN, we proposed estimateQoS API, to inform the developer if during execution another workload takes compute resources away
15:24:00 <anssik> MikeW: WebGPU has a similar timing mechanism
15:24:13 <Mike_Wyrzykowski> https://www.w3.org/TR/webgpu/#timestamp
15:24:59 <anssik> Jonathan: if we look at the algoritm, we want to keep the model simple to minimize overhead, in the future, profiling data could be more comprehensible
15:25:28 <anssik> ... we see some papers suggesting that instead of complex maths, maybe put a simple 1-2 layer network to do the projection
15:25:46 <anssik> ... you do this either via math interpolation or linear neural network
15:25:59 <anssik> ... I'll be exploring this further
15:26:14 <anssik> q?
15:27:03 <anssik> Subtopic: Effective MLComputePolicy
15:27:08 <anssik> Anssi: issue #934
15:27:09 <gb> https://github.com/webmachinelearning/webnn/issues/934 -> Issue 934 Effective MLComputePolicy exposure (by anssiko) [policy selection]
15:27:36 <anssik> ... A number of proposals were shared in this issue, in no particular order:
15:27:49 <anssik> ... - 1) compilation metrics & runtime estimates by MikeW
15:28:10 <mtavenrath> mtavenrath has joined #webmachinelearning
15:28:11 <anssik> ... - 2) low latency v high throughput tradeoff implications by Dwayne
15:28:22 <anssik> ... - 3) strict hints to fail at build by MarkusH, Reilly to speak to this?
15:28:37 <anssik> ... - 4) "low-latency" and "precision" hints by Dwayne
15:28:44 <anssik> ... I propose we discuss each of these separately to stay focused
15:28:51 <anssik> ... prefer to start with a use case or rationale/motivation for the proposed design, then discuss the proposal itself
15:28:58 <anssik> -> 1) compilation metrics & runtime estimates https://github.com/webmachinelearning/webnn/issues/934#issuecomment-4515928955
15:28:58 <gb> https://github.com/webmachinelearning/webnn/issues/934 -> Issue 934 Effective MLComputePolicy exposure (by anssiko) [policy selection]
15:29:43 <anssik> MikeW: if the information we're looking for is latency, other factors of that form, we could expose this directly and mitigate fingerprinting concerns by bucketing every 10-20 ms
15:29:57 <anssik> ... I think sites could make decision if they want to use WebNN based on that information
15:30:28 <anssik> Anssi: how is WebGPU experience for the similar API?
15:30:46 <reillyg> q+
15:30:47 <anssik> MikeW: we have seen several sites adopt these metrics
15:30:50 <anssik> q?
15:30:52 <anssik> ack reillyg
15:31:50 <anssik> Reilly: my question to MikeW, WebGPU version of this provides after execution metrics, to apply this to WebNN are you implying we'd provide these metrics as the model is being used, or implementations would benchmark at built time and provide that data?
15:32:02 <anssik> MikeW: estimated metrics seem to be what is desired?
15:32:14 <anssik> ... UA could look at what was generated and make an estimate based on that information
15:32:14 <anssik> q?
15:32:32 <RafaelCintron> q+
15:32:38 <anssik> ack RafaelCintron
15:33:15 <anssik> Rafael: in the issue it was said the OS can change depending on where the model runs
15:33:22 <anssik> ... how reliable is the initial estimate?
15:33:34 <anssik> MikeW: repeatedly calling this could give different estimates
15:33:42 <anssik> ... depending on how UA decides to implement this
15:34:20 <anssik> ... in WebGPU this is part of the render pass
15:34:21 <anssik> q?
15:34:41 <anssik> -> 2) low-latency differs from throughput https://github.com/webmachinelearning/webnn/issues/934#issuecomment-4516188947
15:34:42 <gb> https://github.com/webmachinelearning/webnn/issues/934 -> Issue 934 Effective MLComputePolicy exposure (by anssiko) [policy selection]
15:35:32 <anssik> Dwayne: highest throughput may not be always lowest latency due to the need move data back and forth
15:35:51 <anssik> q?
15:35:53 <Mike_Wyrzykowski> q+
15:35:57 <anssik> ack Mike_Wyrzykowski
15:36:10 <anssik> MikeW: if we did have these metrics, we could have metrics for latency and total runtime duration
15:36:17 <RafaelCintron> q+
15:36:21 <anssik> ack RafaelCintron
15:36:31 <RafaelCintron> https://www.w3.org/TR/webgpu/#dom-gpuqueue-onsubmittedworkdone
15:37:23 <anssik> Rafael: wanted to say WebGPU has onsubmittedworkdone that fires when the task completes
15:37:36 <anssik> ... timer is more precise
15:37:37 <anssik> q?
15:38:19 <anssik> -> 3) strict hints to fail at build https://github.com/webmachinelearning/webnn/issues/934#issuecomment-4533601729
15:38:19 <gb> https://github.com/webmachinelearning/webnn/issues/934 -> Issue 934 Effective MLComputePolicy exposure (by anssiko) [policy selection]
15:39:20 <anssik> Reilly: I think the metrics idea here are good, my main concern in this example of WebNN is that there are so many different paths the UA can take to implement the model, that just looking at latency and throughput may not fully capture the optimization space for the applicaion
15:39:51 <anssik> ... MarkusH's proposal initial had interop concerns, "build this model with there performance standards, otherwise fail"
15:40:40 <anssik> ... the proposal here is that in the process of building, what HW the model is compatible with, the UA can trigger a build failure if the model is not compatible to be executed on low-power or high-performance HW
15:41:04 <anssik> ... proposal to let the UA to decide where to execute the model and pick the HW where to execute
15:41:19 <anssik> ... if not compatible with the HW, signal a failure
15:41:41 <anssik> ... some frameworks provide some level of fallback, so considering whether strict as an option would all the UA to move things around
15:41:57 <anssik> ... "string" may not be the best term, better the model is compatible with a particular execution path
15:42:08 <anssik> s/string/strict
15:42:10 <anssik> q?
15:42:13 <ningxin> q+
15:42:18 <anssik> ack ningxin
15:42:47 <anssik> Ningxin: thanks for the discussion, this reminds me on performance discussion, data moving
15:43:29 <anssik> ... audio suppression use case, want to run on CPU because moving data around is overhead and as a small model no need to move data
15:43:46 <anssik> ... developer may want to hint the developer from where the data comes from and cares about overhead of data moving
15:44:41 <anssik> ... "I want to do compute close to the data" hint may be what helps in such use cases
15:44:48 <anssik> ... IPC also introduced overhead
15:44:50 <anssik> q?
15:45:39 <anssik> Ningxin: pipeline use case, "I want this WebNN context to interop with WebGPU rendering pipeline"
15:46:19 <anssik> ... for audio noise suppression, we need something other than just "fallback", I know CPU is not a good name to use
15:46:30 <anssik> ... want to focus on the existing use cases
15:46:57 <anssik> ... compute policy now focuses on compute (high TOPS), not data moving
15:46:58 <anssik> q?
15:47:07 <anssik> -> 4) "low-latency" and "precision" hints https://github.com/webmachinelearning/webnn/pull/923#issuecomment-4305974696
15:47:08 <gb> https://github.com/webmachinelearning/webnn/pull/923 -> Pull Request 923 Refactor device selection: Rename to computePolicy, remove accelerated, and add fallback (by mingmingtasd) [policy selection]
15:48:12 <ningxin> q+
15:48:14 <anssik> Dwayne: re "low-latency" as Ningxin said, for "precision" I meant another signal for this
15:48:17 <anssik> ack ningxin
15:49:14 <anssik> Ningxin: for "precision", in WebNN we expect the inference engine to adhere to what is defined for the graph, do you mean if we introduce "precision" we allow the inference engine to do low-precision compute, e.g. casting?
15:49:38 <anssik> Dwayne: this does not apply currently, in the future we could have an opt-in to say "I want higher precision to matter"
15:50:08 <anssik> MarkusT: precision is important for trigonometric functions
15:50:34 <anssik> ... if there would be a way to say I'm find chopping off some low bits it could be useful for some models
15:50:35 <anssik> q?
15:51:24 <anssik> Dwayne: by default I'd lean on conformance and allow hints to loosen that expectation via hints
15:51:24 <anssik> q?
15:51:48 <anssik> q?
15:53:10 <zolkis> q+
15:53:15 <anssik> ack zolkis
15:53:45 <anssik> Zoltan: wanted to mention accelerators will evolve and I'd prefer record the use case preference so the implementation can make mapping to the current and future silicon
15:55:42 <anssik> Anssi: would like to gauge whether any of these 4 proposals have strong support from the group, and if so, which one(s) we should prioritize for further exploration and potential implementation?
15:55:51 <anssik> ... feedback welcome via GH on that
15:56:03 <anssik> Subtopic: Dynamic shapes
15:56:08 <anssik> Anssi: issue #883
15:56:08 <gb> https://github.com/webmachinelearning/webnn/issues/883 -> Issue 883 Support flexible input sizes (by huningxin) [feature request] [operator specific]
15:56:21 <anssik> Anssi: we have new dispatch-time shape validation implementation experience delivered by Bin Miao, thank you!
15:56:25 <anssik> -> https://github.com/webmachinelearning/webnn/issues/883#issuecomment-4486385528
15:56:26 <gb> https://github.com/webmachinelearning/webnn/issues/883 -> Issue 883 Support flexible input sizes (by huningxin) [feature request] [operator specific]
15:57:57 <anssik> Ningxin: team implemented a prototype of dispatch-time shape validation
15:58:40 <anssik> ... this means we verify that the intermediate and output shapes generated throughout the entire graph by actual inputs are valid
15:59:10 <anssik> ... four models tested with different input sized
15:59:23 <anssik> ... O(N) complexity with the number of nodes in the graph
16:00:46 <anssik> ... question to the group "Can we compose the equivalent of 0 and -1 with the newly proposed shape and dynamicReshape (or reshapeDynamic)?"
16:01:34 <anssik> ... it would be useful if WebNN allows compute the final graph's shape when they specify the inputs before the dispatch
16:01:36 <anssik> ... question to the group, should we expose the inferred output shapes a caller?
16:01:44 <anssik> s/shapes/shapes to
16:01:45 <anssik> q?
16:03:57 <anssik> Ningxin: the two questions we can resolve via GH issue discussion, we plan to do more model testing and report results to the group
16:04:17 <anssik> ... we know this is a highly sought after feature so we want to make it real
16:04:36 <anssik> q?
16:05:35 <anssik> RRSAgent, draft minutes
16:05:37 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
16:21:50 <anssik> s|daop-slides|https://raw.githubusercontent.com/webmachinelearning/daop/refs/heads/main/presentations/2026-06-04-daop-update.pdf
16:21:52 <anssik> RRSAgent, draft minutes
16:21:54 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
16:28:03 <anssik> s|https://raw.githubusercontent.com/webmachinelearning/daop/refs/heads/main/presentations/2026-06-04-daop-update.pdf|https://github.com/webmachinelearning/daop/blob/main/presentations/2026-06-04-daop-update.pdf
16:28:05 <anssik> RRSAgent, draft minutes
16:28:06 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
16:31:27 <anssik> s/is in one part/is one part
16:32:33 <anssik> s/algoritm/algorithm
16:42:35 <anssik> s/built time/build time
16:43:16 <anssik> s/change depending on where/decide to change where
16:43:36 <anssik> s/need move/need to move
16:44:20 <anssik> s/applicaion/application
16:44:36 <anssik> s/initial had/initially had
16:44:48 <anssik> s/there performance/these performance
16:45:32 <anssik> s/all the/allow the
16:46:12 <anssik> s/the developer from/from
16:47:06 <anssik> s/and cares/and indicate preference
16:47:20 <anssik> s/introduced/introduces
16:48:13 <anssik> s/find/fine
16:48:27 <anssik> s/via hints//
16:49:37 <anssik> s/sought after/sought-after
16:49:46 <anssik> RRSAgent, draft minutes
16:49:48 <RRSAgent> I have made the request to generate https://www.w3.org/2026/06/04-webmachinelearning-minutes.html anssik
17:10:34 <anssik> anssik has joined #webmachinelearning
17:10:34 <chrishtr> chrishtr has joined #webmachinelearning
17:10:34 <christianliebel> christianliebel has joined #webmachinelearning
17:10:34 <hyojin> hyojin has joined #webmachinelearning
17:10:34 <gregwhitworth> gregwhitworth has joined #webmachinelearning
17:10:34 <vmpstr> vmpstr has joined #webmachinelearning
17:10:34 <tomayac27> tomayac27 has joined #webmachinelearning
17:10:34 <jyasskin> jyasskin has joined #webmachinelearning
17:10:34 <awafaa> awafaa has joined #webmachinelearning
17:10:34 <vasilii> vasilii has joined #webmachinelearning
18:14:32 <anssik> anssik has joined #webmachinelearning
18:14:32 <chrishtr> chrishtr has joined #webmachinelearning
18:14:32 <christianliebel> christianliebel has joined #webmachinelearning
18:14:32 <hyojin> hyojin has joined #webmachinelearning
18:14:32 <gregwhitworth> gregwhitworth has joined #webmachinelearning
18:14:32 <vmpstr> vmpstr has joined #webmachinelearning
18:14:32 <tomayac27> tomayac27 has joined #webmachinelearning
18:14:32 <jyasskin> jyasskin has joined #webmachinelearning
18:14:32 <awafaa> awafaa has joined #webmachinelearning
18:14:32 <vasilii> vasilii has joined #webmachinelearning
18:26:38 <Zakim> Zakim has left #webmachinelearning