IRC log of webmachinelearning on 2024-09-05

Timestamps are in UTC.

13:56:11 [RRSAgent]
RRSAgent has joined #webmachinelearning
13:56:16 [RRSAgent]
logging to https://www.w3.org/2024/09/05-webmachinelearning-irc
13:56:16 [anssik]
Meeting: WebML WG Teleconference – 5 September 2024
13:56:16 [Zakim]
RRSAgent, make logs Public
13:56:17 [Zakim]
please title this meeting ("meeting: ..."), anssik
13:56:17 [anssik]
Chair: Anssi
13:56:21 [anssik]
Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2024-09-05-wg-agenda.md
13:58:56 [anssik]
Scribe: Anssi
13:59:00 [anssik]
scribeNick: anssik
13:59:09 [anssik]
gb, this is webmachinelearning/webnn
13:59:09 [gb]
anssik, OK.
13:59:13 [ningxin]
ningxin has joined #webmachinelearning
13:59:15 [anssik]
RRSAgent, draft minutes
13:59:17 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
14:01:29 [RafaelCintron]
RafaelCintron has joined #webmachinelearning
14:01:32 [anssik]
Present+ Joshua_Lochner
14:01:33 [Mike_Wyrzykowski]
Mike_Wyrzykowski has joined #webmachinelearning
14:01:38 [anssik]
Present+ Michael_McCool
14:01:42 [anssik]
Present+ Ningxin_Hu
14:01:53 [anssik]
Present+ Mike_Wyrzykowski
14:02:03 [anssik]
Present+ Austin_Sullivan
14:02:05 [dom]
dom has joined #webmachinelearning
14:02:24 [anssik]
Present+ Dwayne_Robinson
14:02:51 [anssik]
Present+ Dominique_Hazael-Massieux
14:02:54 [asully]
asully has joined #webmachinelearning
14:03:53 [anssik]
RRSAgent, draft minutes
14:03:54 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
14:04:50 [anssik]
Topic: TPAC F2F agenda published
14:05:16 [gb]
Issue 180 not found
14:05:18 [anssik]
-> https://github.com/webmachinelearning/meetings/issues/25
14:05:18 [gb]
https://github.com/webmachinelearning/meetings/issues/25 -> Issue 25 WebML WG - TPAC 2024 agenda (by anssiko)
14:05:59 [anssik]
anssik: TPAC F2F agenda has been published. We'll do fine-tuning and gardening: confirm our topic leads, timeboxes, discuss group contributions that'd be welcome by F2F.
14:07:13 [anssik]
anssik: current F2F plan is as follows
14:07:34 [McCool]
McCool has joined #webmachinelearning
14:07:45 [anssik]
... Start meeting 9:00 local / 16:00 UTC
14:08:04 [Joshua_Lochner]
Joshua_Lochner has joined #webmachinelearning
14:08:42 [anssik]
... 15-sec WG participant intros: name, affiliation, top interest [10 mins]
14:08:50 [anssik]
... currently 20 group participants and 17 observers registered in person
14:09:02 [anssik]
... Agenda bashing for last-minute drops/adds, not much wiggle room to add things [5 mins]
14:09:25 [anssik]
... Charter orientation mainly to inform observers and folks from other WGs [5 mins]
14:09:58 [anssik]
Subtopic: Ethics
14:10:16 [anssik]
anssik: - 9:30-10:00 local / 16:30-17:00 UTC in-browser explainability libraries and viz tools (Jay Wang, OpenAI / Georgia Tech) [30 mins]
14:10:22 [anssik]
... for a sneak-peek, see Jay's PhD defense:
14:10:32 [anssik]
-> Democratizing Human-Centered AI with Visual Explanation and Interactive Guidance https://zijie.wang/phd-thesis/
14:11:15 [anssik]
Subtopic: Spec orientation [30 mins]
14:12:03 [anssik]
anssik: - Triage pass through open issues: breaking changes, priorities, next steps for the issue (All)
14:12:10 [anssik]
... to prep for this triage exercise, please bring with you your favorite issues and prepare to share your suggestions for next steps
14:12:14 [anssik]
... focus on issues w/o agenda time already allocated
14:12:19 [anssik]
... first come, first served
14:12:40 [anssik]
-> Triage guidance https://github.com/webmachinelearning/webnn/blob/main/docs/IssueTriage.md
14:13:46 [anssik]
Subtopic: New features
14:14:05 [anssik]
... - A refreshed analysis of popular models, operator & data type gaps
14:14:12 [anssik]
... Dwayne would you be able to present a summary that fits into 20 mins?
14:14:46 [anssik]
Dwayne: 15 minutes is good
14:15:25 [anssik]
... - Quantization and dequantization (QDQ), a bag of multiple issues
14:16:15 [anssik]
Austin: someone from Google can present this
14:17:42 [anssik]
anssik: - Platform capability detection
14:17:57 [anssik]
... spec definition and prototype done!
14:18:04 [anssik]
... discuss impl experience, deferred rankRange feature
14:19:00 [anssik]
Austin: someone from Google side can talk to this, 5 min, TBD if needs discussion
14:19:23 [anssik]
... - Future-proof device selection abstractions
14:19:36 [anssik]
s/... -/anssik: ... -
14:20:10 [anssik]
... MikeW, can prepare a concrete proposal that'd work for WebKit for the group to discuss?
14:20:29 [anssik]
Mike: of course, can talk to this issue and provide a proposal, 5 minutes should be fine
14:21:08 [anssik]
Subtopic: Customer feedback & collaborations
14:21:17 [anssik]
anssik: 13:30-14:00 local - WebLLM and MLC
14:21:28 [anssik]
... Tianqi Chen to share with us the latest from the WebLLM, MLC, Apache TVM
14:21:40 [anssik]
... he was interested in WebNN-WebGPU interop progress, so good to share that with him
14:22:26 [anssik]
... - Transformers.js WebNN backend
14:24:07 [anssik]
JoshuaL: will attend virtually, will showcase Transformers.js WebNN demo, EP selection via ONNX Runtime, demo in the works with up-to-date visual models
14:25:10 [anssik]
... 10-15 mins user-centric demonstration, not too low-level on the details
14:25:54 [anssik]
... - ONNX Runtime Web & WebNN EP
14:26:06 [anssik]
JoshuaL: can ask Gunther if he could present on this topic
14:26:22 [anssik]
... 10-15 mins
14:26:37 [anssik]
anssik: - Google Chrome Feedback revisited #453
14:26:38 [gb]
https://github.com/webmachinelearning/webnn/issues/453 -> Issue 453 Google Chrome Feedback on WebNN: aiming for broad device coverage and maintainability (by vsekhar) [process] [opset] [use case]
14:27:57 [anssik]
Austin: a lot of the points in this are overlapping with other issues, so we will check if we should keep this on the agenda
14:28:06 [anssik]
... not need a lot of time regardless
14:28:22 [anssik]
Subtopic: Interop and cross-group coordination
14:28:33 [anssik]
... - Interop issues across different backends
14:28:36 [anssik]
... Ningxin, how much time to allocate for this?
14:28:43 [ningxin]
https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Ainterop
14:29:11 [anssik]
ningxin: I want to work through issues labeled with "interop", some are covered in "device selection", ConstantOperand also has a dedicated issue
14:29:32 [anssik]
... int64 is one interesting issue with MikeW feedback
14:29:52 [anssik]
... zero-sized dimensions
14:30:07 [anssik]
... want to triage all "interop" issue for discussion
14:30:43 [anssik]
... - Core operator #573
14:30:43 [gb]
https://github.com/webmachinelearning/webnn/issues/573 -> Issue 573 Core operator set (by philloooo) [question] [opset]
14:30:52 [anssik]
s/... - /anssik: -
14:31:13 [anssik]
Austin: related to custom op incubation, we can merge this with custom op
14:31:47 [anssik]
ningxin: we may reveal other options in incubation section, core op set is a good foundation
14:32:05 [anssik]
Austin: we can have a few minutes for this
14:33:08 [anssik]
... - MLTensor #754
14:33:09 [gb]
https://github.com/webmachinelearning/webnn/pull/754 -> Pull Request 754 Add MLTensor explainer (by a-sully) [webgpu interop]
14:33:26 [anssik]
... Austin, possibly Corentin from WebGPU joining
14:33:50 [anssik]
s/... - /anssik: -
14:34:09 [anssik]
... - MLConstantOperand #668
14:34:09 [gb]
https://github.com/webmachinelearning/webnn/issues/668 -> Issue 668 Do we need an `MLConstantOperand`? (by a-sully) [question] [interop]
14:34:21 [anssik]
Subtopic: Implementation plans and trials
14:34:33 [anssik]
anssik: Next step for implementations, Origin Trial or equivalent and align with framework developer feedback
14:35:03 [anssik]
ningxin: we're at CR "call for implementations", use this as an opportunity to get signals from implementers
14:35:25 [anssik]
Subtopic: Advancement on the W3C Rec Track
14:35:52 [anssik]
anssik: - Wide review status, close on TAG review feedback
14:36:08 [anssik]
... - W3C “living standards” expectations and plan with Dom
14:36:35 [anssik]
Dom: personally would suggest to not go to living CR model
14:36:36 [Joshua_Lochner]
Joshua_Lochner has joined #webmachinelearning
14:37:00 [anssik]
Subtopic: Incubation
14:37:05 [anssik]
anssik: - Custom ops
14:37:18 [anssik]
... Ningxin, 20 mins
14:37:29 [anssik]
... - Built-in APIs for translation and prompting
14:37:38 [anssik]
... Dominic, ~20 mins
14:37:55 [anssik]
... - Model management
14:38:01 [anssik]
... Michael McCool, ~10-15 mins
14:38:15 [anssik]
McCool: will have an additional breakout session
14:38:23 [anssik]
Subtopic: Wrap up & Dinner
14:38:39 [anssik]
anssik: our F2F group is quite big so we're not able to book one restaurant
14:38:40 [dom]
-> https://github.com/w3c/tpac2024-breakouts/issues?q=is%3Aissue+is%3Aopen+label%3A%22track%3A+AI%22 AI-related TPAC breakouts
14:38:44 [anssik]
... one option would be to go to Anaheim Packing District 2.5 miles from the meeting venue
14:38:54 [anssik]
... "There are several ways to enjoy the culinary artistry from our small businesses, including takeout, delivery, indoor dining and outdoor dining."
14:39:05 [anssik]
... a wide range of options so could split by food preference
14:39:14 [anssik]
... if some Anaheim local foodie has recommendations, please share!
14:39:20 [anssik]
-> https://www.anaheimpackingdistrict.com/merchants
14:40:06 [RafaelCintron]
RafaelCintron has joined #webmachinelearning
14:40:18 [anssik]
Topic: Device selection abstractions
14:40:47 [anssik]
anssik: We agreed to evolve MLContextOptions and other API controls for device selection informed by further implementation experience and new use cases from the wider web community.
14:40:51 [anssik]
... I wanted to discuss how frameworks and backends approach configuring inference sessions, understand framework-level use cases and requirements. Distill learnings to help evolve MLContextOptions.
14:40:56 [anssik]
... ONNX Runtime Web does provide some knobs, which vary by backend
14:41:00 [anssik]
-> Session Options API https://onnxruntime.ai/docs/api/js/interfaces/InferenceSession.SessionOptions.html
14:41:06 [anssik]
anssik: for example:
14:41:15 [anssik]
... executionMode?: "sequential" | "parallel" - Controls whether multiple operators in the graph across nodes run sequentially or in parallel
14:41:26 [anssik]
... graphOptimizationLevel?: "basic" | "all" | "disabled" | "extended" - graph-level transformations, ranging from small graph simplifications and node eliminations to more complex node fusions and layout optimizations
14:41:32 [anssik]
... there's also WebNN EP Options
14:41:42 [anssik]
-> WebNN EP Session Options API https://onnxruntime.ai/docs/api/js/types/InferenceSession.WebNNExecutionProviderOption.html
14:41:52 [anssik]
anssik: WebNNExecutionProviderOption: WebNNOptionsWithoutMLContext | WebNNOptionsWithMLContext | WebNNOptionsWebGpu
14:41:58 [anssik]
... interface WebNNOptionsWithMLContext extends MLContextOptions with numThreads
14:42:02 [anssik]
... I wanted to understand whether all these knobs are purely ONNX implementation details or whether some of these could be WebNN MLContextOptions hints.
14:42:15 [anssik]
JoshuaL: would like to have Gunther involved, he'd be an expert in this
14:42:29 [anssik]
... number of thread user for Wasm EP, proxy by WebWorker
14:42:48 [anssik]
... I talked with Gunther that ORT docs are not up to date
14:43:17 [anssik]
... for Transformers.js we set some good defaults, set num of threads
14:44:19 [anssik]
anssik: I also wanted to remind us of CoreML limitations and on what is implementable on Apple platforms, see issue #749 where MikeW provided information about CoreML MLComputeUnits configuration limitations
14:44:19 [gb]
https://github.com/webmachinelearning/webnn/issues/749 -> Issue 749 MLContextOptions.deviceType seems unnecessary outside of conformance testing (by mwyrzykowski) [device selection]
14:44:23 [anssik]
-> https://developer.apple.com/documentation/coreml/mlcomputeunits/
14:44:28 [Joshua_Lochner]
Here's the env: https://github.com/microsoft/onnxruntime/blob/main/js/common/lib/env.ts
14:44:45 [anssik]
anssik: the current WebNN API device selection mechanism we declared subject to change is:
14:44:49 [anssik]
-> Device selection https://www.w3.org/TR/webnn/#programming-model-device-selection
14:44:53 [anssik]
-> MLContextOptions https://www.w3.org/TR/webnn/#dictdef-mlcontextoptions
14:45:24 [anssik]
MikeW: not closely following framework configuration options, but will check this ahead of TPAC discussion
14:45:42 [anssik]
q?
14:46:02 [anssik]
Topic: MLTensor
14:46:06 [anssik]
anssik: MLTensor explainer PR #754 received a lot of great feedback and review, thank you!
14:46:06 [gb]
https://github.com/webmachinelearning/webnn/pull/754 -> Pull Request 754 Add MLTensor explainer (by a-sully) [webgpu interop]
14:46:09 [anssik]
-> MLTensor Explainer (preview) https://github.com/webmachinelearning/webnn/blob/d3e2be575d1879c7c7cd5f438ffc6b6f7d7b3c30/mltensor-explainer.md
14:46:15 [anssik]
Austin: thanks for all the feedback!
14:46:58 [anssik]
... re Corentin's feedback, MLTensor makes an assumption the MLTensorOptions whether readable from script (or not) is sufficient to know where to allocate the tensor on the system
14:47:12 [anssik]
... open questions if this is true, if device selection is tied to context
14:47:33 [anssik]
... Corentin pointed out there's issues with making GPU wait on a fence from another place in the system
14:47:52 [anssik]
... there's still some open questions, with more clarify on device selection not sure how much progress we can make
14:48:45 [anssik]
anssik: it looks like GH feedback from Rafael, Dwayne, Ningxin addressed
14:48:58 [anssik]
... GH feedback from Bryan, Phillis and Corentin is still being worked on
14:49:12 [anssik]
anssik: merging an explainer is lower bar than merging a spec PR
14:49:21 [anssik]
Austin: will add notes for Corentin's feedback in the explainer
14:49:32 [anssik]
Dom: what would be the next step before getting this into the spec prose and IDL?
14:49:37 [anssik]
... is it device selection clarification?
14:49:47 [anssik]
Austin: we need to specify timelines
14:49:55 [anssik]
... especially for WebGPU interop
14:50:21 [anssik]
... with WebGPU has its own concept of timelines and need to define how WebNN interacts with that timeline
14:50:42 [anssik]
... when WebNN rents MLTensor WebNN timeline needs to wait when returned
14:50:55 [anssik]
... WebNN timeline relationship with WebGPU timeline
14:51:23 [anssik]
... for context, Chai had an earlier PR that said WebNN's timelime is same as WebGPU CommandBuffer
14:51:34 [anssik]
... that matches DirectML model, but not CoreML, TFLite
14:51:49 [anssik]
... from spec perspective timelines must be a bit more decoupled
14:52:14 [anssik]
q?
14:53:01 [anssik]
McCool: I was planning to prototype caching based on MLBuffer/MLTensor, weights from model that can be restored
14:53:33 [anssik]
Austin: MLTensor used for input and outputs, the challenge is, sharing weights across models is not something that is supported by DirectML
14:56:03 [anssik]
RRSAgent, draft minutes
14:56:04 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
14:56:56 [dom]
-> https://www.w3.org/wiki/MeetingTaxis Meeting Taxis for TPAC 2024
14:57:46 [anssik]
q?
14:57:56 [anssik]
Topic: See you at TPAC F2F!
14:58:04 [anssik]
anssik: see you at TPAC F2F on 23 September 2024 for a day of exciting discussions and more!
14:58:11 [anssik]
... I'll cancel the bi-weekly telcons that have close proximity with the TPAC week:
14:58:15 [anssik]
... - cancel WebML WG Teleconference – 19 September 2024
14:58:22 [anssik]
... - cancel WebML WG Teleconference – 3 October 2024
14:58:33 [anssik]
... Our next call after TPAC will be 17 October 2024
14:58:35 [anssik]
RRSAgent, draft minutes
14:58:36 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
15:00:36 [anssik]
s/... - A refreshed/anssik: - A refreshed
15:00:38 [anssik]
RRSAgent, draft minutes
15:00:39 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
15:01:04 [anssik]
s/Spec orientation [30 mins]/Spec orientation
15:01:06 [anssik]
RRSAgent, draft minutes
15:01:07 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
15:02:00 [anssik]
s/Dominic/Domenic
15:03:41 [anssik]
s/user for/used for
15:05:36 [anssik]
s/with WebGPU has/WebGPU has
15:09:08 [anssik]
s/not need/do not need
15:13:26 [anssik]
s/thread user/threads used
15:19:55 [anssik]
RRSAgent, draft minutes
15:19:56 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
15:21:08 [anssik]
s/number of thread/number of threads
15:21:20 [anssik]
RRSAgent, draft minutes
15:21:22 [RRSAgent]
I have made the request to generate https://www.w3.org/2024/09/05-webmachinelearning-minutes.html anssik
17:17:11 [Zakim]
Zakim has left #webmachinelearning