14:58:58 RRSAgent has joined #webmachinelearning 14:59:02 logging to https://www.w3.org/2023/11/30-webmachinelearning-irc 14:59:02 RRSAgent, make logs Public 14:59:03 please title this meeting ("meeting: ..."), anssik 14:59:06 Meeting: WebML WG Teleconference – 30 November 2023 14:59:11 Chair: Anssi 14:59:15 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2023-11-30-wg-agenda.md 14:59:19 Scribe: Anssi 14:59:23 scribeNick: anssik 14:59:30 gb, this is webmachinelearning/webnn 14:59:30 anssik, OK. 14:59:35 Present+ Anssi_Kostiainen 14:59:43 chai has joined #webmachinelearning 14:59:53 Present+ Chai_Chaoweeraprasit 14:59:59 Present+ Zoltan_Kis 15:00:13 Present+ Rachel_Yager 15:00:19 RRSAgent, draft minutes 15:00:20 I have made the request to generate https://www.w3.org/2023/11/30-webmachinelearning-minutes.html anssik 15:00:28 etiennenoel has joined #webmachinelearning 15:00:28 Regrets+ Dominique_Hazael-Massieux 15:00:30 jsbell has joined #webmachinelearning 15:00:30 Present+ Etienne_Noel 15:00:52 Present+ Joshua_Bell 15:01:09 Ningxin_Hu has joined #webmachinelearning 15:02:00 Present+ Joshua_Lochner 15:02:07 Present+ Rafael_Cintron 15:02:22 Present+ Dwayne_Robinson 15:02:30 Joshua_Lochner has joined #webmachinelearning 15:02:57 Topic: Announcements 15:03:01 Subtopic: Implementation status 15:03:06 -> Implementation Status of WebNN Operations https://webmachinelearning.github.io/webnn-status/ 15:03:10 anssik: implementation status has been updated for: 15:03:19 ... - WebNN CPU XNNPack backend 15:03:19 ... - WebNN GPU DirectML backend 15:03:19 ... - ORT 15:03:23 ... for details, please see the webnn_status.json diff at https://github.com/webmachinelearning/webmachinelearning.github.io/pull/58/files 15:03:30 anssik: this was a team effort, thanks @lisa0314 @miaobin @Honry @mingmingtasd @BruceDai @shiyi9801! 15:03:30 https://github.com/lisa0314 -> @lisa0314 15:03:30 … https://github.com/miaobin -> @miaobin 15:03:30 … https://github.com/Honry -> @Honry 15:03:30 … https://github.com/mingmingtasd -> @mingmingtasd 15:03:31 … https://github.com/BruceDai -> @BruceDai 15:03:34 … https://github.com/shiyi9801 -> @shiyi9801 15:03:42 RafaelCintron has joined #webmachinelearning 15:04:03 RachelY has joined #webmachinelearning 15:04:03 Subtopic: Upcoming discussion with the Web LLM author Tianqi Chen 15:04:17 dwayner has joined #webmachinelearning 15:04:19 anssik: I had a discussion with Tianqi Chen from CMU, OctoML, creator of Web LLM, a JS library that accelerates select LLMs in browsers with WebGPU 15:04:27 -> Web LLM repo https://github.com/mlc-ai/web-llm 15:04:35 ... Tianqi shared he's very supportive of our work in this WG and is interested in working with us more closely 15:04:47 ... given Tianqi's highly relevant expertise and interest, I've initiated the process to bring him on board the WG as an Invited Expert 15:04:52 ... this will allow him to contribute in a full capacity 15:04:58 ... I've tentatively scheduled a WG discussion with Tianqi on 11 January 2023 15:05:08 ... Tianqi has already shared use cases with this WG for: 15:05:11 ... - hybrid execution of models i.e. WebGPU for custom ops + WebNN) 15:05:26 ... - JSON schema of the webNN declaration i.e. the compiler projects can generate a schema and invoke executions without explicitly doing so in JS 15:05:39 ... if you have questions to Tianqi e.g. about his work on WebLLM that will be a great opportunity to ask those questions 15:05:43 ... we have opened a dedicated issue for the hybrid execution use case, currently a high-level description, but can be appended to with more details 15:05:46 -> Hybrid execution use case issue https://github.com/webmachinelearning/webnn/issues/480 15:05:50 https://github.com/webmachinelearning/webnn/issues/480 -> Issue 480 Hybrid execution use case from Web LLM project (by anssiko) [use case] 15:06:10 Topic: WebNN v2: Review transformer ops spec contributions (continued) 15:06:15 anssik: issue #375 and PR #478 15:06:16 https://github.com/webmachinelearning/webnn/issues/478 -> Pull Request 478 Add support for operations needed for well-known transformers e.g. Segment Anything, Stable Diffusion, etc. (by wchao1115) 15:06:16 https://github.com/webmachinelearning/webnn/issues/375 -> Issue 375 Support for transformers (by dontcallmedom) [v2] [operation set] 15:06:32 ... on our last call I drew the WG's attention to this major PR #478 that was looking for everyone’s review and feedback 15:07:04 ... I also shared my expectation that by this meeting on 30 Nov we are in a position to make a merge decision, let's discuss now whether we're there yet or whether we want some additional time for further refinements 15:07:22 ... first, I want to thank the the entire group for your active review and Chai for responding to the review comments that reflect the group' consensus 15:07:28 ... this major PR has been a great group effort, thank you all! 15:07:42 ... I produced a hand-rolled IDL diff between the latest published version from 26 October 2023 and this PR 15:07:45 -> Hand-rolled IDL diff https://github.com/webmachinelearning/webnn/pull/478#issuecomment-1833600693 15:08:19 ... I compiled a list of opens from the PR review, some of these may require no action, so we can go through quickly 15:08:45 zkis has joined #webmachinelearning 15:08:45 Subtopic: NavigatorML mixin 15:08:51 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1410542879 15:09:04 anssik: I suppose the NavigatorML mixin was removed by accident, without this mixin, the ML object is not exposed via navigator.ml 15:09:10 ... a fix is to bring this back 15:10:54 chai: thanks for the review everyone, especially Ningxin and Dwayne for careful comments 15:11:13 ... all reviewers please resolve discussions in the GH PR that have been resolved 15:11:56 ... for changes unrelated to this PR, please open a separate issue and link to this big PR 15:14:19 Subtopic: Hard to tell whether an MLOperand is a constant or not 15:14:26 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1410103395 15:14:31 -> https://www.w3.org/TR/webnn/#dom-mlgraphbuilder-constant 15:14:53 Ningxin: It's hard to tell whether an MLOperand is a constant or not. The current algorithm of constant only creates an implementation-defined platform operand and sets values to it. 15:15:01 q+ 15:15:43 ack Ningxin_Hu 15:16:06 Ningxin_Hu: this is for the gather op validating index parameter 15:17:09 ... a step in the validation algorithm, unless constant the implementation can access the data of that operand, otherwise it is runtime behavior 15:18:10 ... current algorithm step only works with constant operation, Chai wanted me to propose some text to address this, we don't mark op as constant or not, need some text ensure 15:19:00 ... in Chromium impl discussion on how to address OOB, proposed for discussion in a separate issue 15:19:43 Chai: I think this should be tracked as a separate issue 15:19:55 Zoltan: platform tests for this that it is always a constant? 15:20:06 Ningxin_Hu: it does not need to always be constant I think 15:20:49 anssik: proposal to create a separate issue for this 15:20:53 Ningxin_Hu: I'll do that 15:20:56 anssik: thanks! 15:21:06 Subtopic: Make standalone argMax and argMin operators 15:21:15 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1410104309 15:21:21 -> https://pr-preview.s3.amazonaws.com/webmachinelearning/webnn/pull/478.html#mlgraphbuilder-reduce-op 15:21:39 Ningxin: two issues of output operand creation: 15:21:46 ... - The output shape should be calculated instead of just copying input's. 15:21:59 ... - The output data type of reduceArgMax and reduceArgMin should be unsigned integer rather than setting to input's. 15:22:34 Chai: I agree these should be separated out 15:23:05 ... I will commit that change soon to this branch 15:23:28 Subtopic: Gather op implementation considerations for out-of-bound indices 15:23:33 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1396672041 15:23:44 anssik: Jiewei has a proposal for an informative section: 15:23:53 ... 1. Runtime out-of-bounds indices should be explicitly handled by either the browser implementation or the platform implementation, to avoid OOB memory accesses. 15:23:53 ... 2. If the platform implementation doesn't handle out-of-bounds indices, the browser implementation should take steps to ensure the platform operator doesn't receive out-of-bound indices 15:24:35 anssik: I propose these bullets to be added in gather section as an informative note and a link added to https://www.w3.org/TR/webnn/#security 15:24:48 ... for the third bullet, a separate GH issue should be opened: 15:24:52 ... 3. Mention what caller should expect as a result of list item 2: 0, NaN, first/last indices (if implemented with clamp) 15:25:46 Dwayne: discussing this in context of the separate gather issue sounds good 15:26:22 Subtopic: MLReduceOptions.keepDimensions scan direction 15:26:27 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1396829866 15:26:31 -> https://pr-preview.s3.amazonaws.com/webmachinelearning/webnn/pull/478.html#dictdef-mlreduceoptions 15:26:51 anssik: Dwayne notes that for PT/TF compat, should support both increasing and decreasing axis scan directions for tied values 15:27:08 ... this issue preexists this PR, proposed as a separate issue 15:28:19 Ningxin_Hu: this is related to reduceArgMax and reduceArgMin, they don't have separate signatures 15:28:40 Dwayne: will discuss with Chai to come up with a solution 15:28:50 q? 15:28:59 Subtopic: Rename MLOperand.type() to dataType() 15:29:08 -> https://github.com/webmachinelearning/webnn/pull/478#pullrequestreview-1745700960 15:29:12 -> https://www.w3.org/TR/webnn/#dom-mloperanddescriptor-datatype 15:29:19 anssik: to align with MLOperandDescriptor.dataType 15:29:32 Chai: already fixed 15:29:47 Subtopic: Examples of how gather works in different slicing schemes 15:29:54 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1402915672 15:29:58 -> https://pr-preview.s3.amazonaws.com/webmachinelearning/webnn/pull/478.html#example-3d538e0c 15:30:02 anssik: Jiewei proposes a more extreme example as the last one 15:30:07 input.shape = (2,3,2) 15:30:07 indices.shape = (3,4,5) 15:30:07 output.shape = (2,3,4,5,2) 15:30:35 Chai: for gather there are quite a few samples, all major cases covered probably 15:30:58 Subtopic: Naming logicalNot or not 15:31:04 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1406997415 15:31:30 Dwayne: OK to make this a separate issue 15:31:50 Subtopic: Naming where arguments 15:31:55 -> https://github.com/webmachinelearning/webnn/pull/478#discussion_r1408773430 15:32:10 anssik: proposal from CL review, rationale "both input and other are 'inputs'" 15:32:17 ... From: MLOperand where(MLOperand condition, MLOperand input, MLOperand other); 15:32:21 ... To: MLOperand where(MLOperand condition, MLOperand trueValue, MLOperand falseValue); 15:33:31 q+ 15:33:38 Chai: naming arguments is hard 15:33:40 q? 15:33:50 ack jsbell 15:34:46 Chai: discoverability is important, in that if someone implements this API in their framework they need to be able to find the thing and map it to their implementation 15:34:54 q- 15:35:39 Subtopic: Merge readiness check 15:36:19 q+ 15:36:45 ack jsbell 15:38:06 jsbell: regarding resolving comments in the PR review, my comments are addressed, I just couldn't resolve those (due to GH permissions thing) 15:38:33 ... looks like this PR tries to resolve all in one PR, rather than multiple smaller PRs 15:39:05 q? 15:39:41 +1 to have small PRs after this one 15:39:44 Chai: agree we should do incremental PRs going forward 15:40:21 q+ 15:41:04 anssik: once all PR review discussions have been resolved we are ready to merge the PR, agreed? 15:41:29 ... either by spinning off into separate open issues or resolving discussion on the spot in the PR 15:42:29 q? 15:42:44 Dwayne: I don't see resolve conversation button either 15:43:54 anssik: please thumb up on the very last comment to signal you're OK to resolve 15:44:28 q? 15:44:30 ack Ningxin_Hu 15:44:41 https://github.com/webmachinelearning/webnn/pull/478#issuecomment-1815864650 15:44:47 Ningxin_Hu: to respond to jsbell re conformance tests, see the link above 15:45:00 ... the baseline implementation is pure JS 15:45:18 ... we have WIP wpt tests too for these ops 15:45:29 ... I hope that will unblock this PR 15:45:50 jsbell: I saw the table, no strong opinion where this lives, someplace where it is maintained properly 15:46:06 Ningxin_Hu: I'll move the table to a separate issue 15:46:22 q? 15:46:31 q+ 15:46:34 ack chai 15:47:13 chai: I have a separate ask, I think that at some point the spec should have a way for the people to identify what are the ops they are talking about 15:47:28 ... the browser needs to identify the implementation to be compliant to something 15:48:00 ... in the early HTML days, there was a notion of CSS Layer 1 and Layer 2, sounded dubious in the beginning, don't know what it might look like here 15:48:01 q? 15:48:18 anssik: Living Standard is the trend 15:49:18 jsbell: there's no perfect answer, one approach some Chrome DevRel folks are working on is called baseline, that's not at the level of individual methods etc. but saying in 2024 there APIs work across browsers 15:49:39 ... not sure if they've looked at lower level "there 5 methods are supported in 2024 across X, Y and Z" 15:50:21 ... the v1 of the spec, with multiple implementations across browsers, we want that browsers only claim compliance when they pass all wpt tests 15:50:50 ... approaches in my specs, I've called things that are new and track as implementations adopt those 15:51:07 ... new functionality could be advertised and communicate what is implemented and where 15:51:19 https://webmachinelearning.github.io/webnn-status/ 15:52:15 jsbell: some non-Chromium browsers often say they have no support for a feature unless they pass all the wpt tests 15:52:40 ... sometimes tests miss something obvious, so this model fails in that aspect sometimes 15:53:02 ... good wpt coverate is important for our team 15:53:04 q? 15:53:42 https://github.com/webmachinelearning/meetings/blob/main/telcons/2023-11-30-wg-agenda.md 15:54:02 zkis_ has joined #webmachinelearning 15:54:05 q? 15:54:57 Subtopic: Should scale and bias be required inputs for batchNormalization op? 15:54:57 anssik: issue #481 15:54:57 https://github.com/webmachinelearning/webnn/issues/481 -> Issue 481 Should `scale` and `bias` be required inputs for `batchNormalization` op? (by huningxin) 15:54:58 anssik: Ningxin did a very thorought investigation into this issue, summary: 15:55:10 ... - currently in batchNormalization scale and bias operands are optional members of MLBatchNormalizationOptions dictionary 15:55:30 ... - current algorithm: if scale is not present, the element-wise multiplication can be eliminated, and if bias is not present, the element-wise addition can be eliminated too 15:55:56 anssik: Ningxin notes, however, there's an issue: "the optional scale and bias are not widely supported across frameworks and native ML APIs. This would cause the implementation more complex for those native ML APIs which don't support optional scale and bias" 15:56:30 ... for details of the framework and native MP APIs, see the GH issue 15:56:30 ... Ningxin proposes a solution: "make the two operands required" 15:56:30 ... and notes that models that won't use scale and bias the frameworks can set scale to 1 and bias to 0 15:56:38 q? 15:56:51 Dwayne: makes sense, don't know why these were originally optional 15:57:02 q? 15:57:10 q+ 15:57:13 ack jsbell 15:57:35 jsbell: I'm relaying a comment from someone from my team who pointed out there may be some confusion re optional and required 15:58:14 ... if we give scale and bias default values, do we want to force developers to pass values? Or use common defaults? 15:58:16 q? 15:58:59 ... i.e. make default scale be 1 and bias be 0 15:59:12 Dwayne: these will be called from frameworks with tensors lying around 15:59:31 Chai: I need some time to read this issue one more time 15:59:56 q+ 15:59:56 ... the idea of optional in the API is to not make the API signature cluttered and help with future revisions 16:00:13 ... for this specific issue, scale and bias as required, need to require how strong this feedback is 16:00:24 ... you have something optional, you can always ask to be more explicit about it 16:00:34 ack Ningxin_Hu 16:01:04 Ningxin_Hu: SGTM, I think there's an opportunity to make this an optimization opportunity 16:01:26 ... if some native ML API can make use of this optimization then keeping this optional is reasonable 16:01:31 ... open for discussion 16:01:40 q? 16:01:40 I'll discuss in more detail w/ my folks, see if they want to add comments to the issue 16:01:47 RRSAgent, draft minutes 16:01:48 I have made the request to generate https://www.w3.org/2023/11/30-webmachinelearning-minutes.html anssik 16:03:11 q? 16:07:15 s|https://github.com/webmachinelearning/meetings/blob/main/telcons/2023-11-30-wg-agenda.md|Topic: Enhancements 16:07:18 RRSAgent, draft minutes 16:07:20 I have made the request to generate https://www.w3.org/2023/11/30-webmachinelearning-minutes.html anssik 16:10:26 s|@shiyi9801|@shiyi9801 @ibelem 16:10:26 https://github.com/shiyi9801 -> @shiyi9801 16:10:26 … https://github.com/ibelem -> @ibelem 16:10:31 RRSAgent, draft minutes 16:10:32 I have made the request to generate https://www.w3.org/2023/11/30-webmachinelearning-minutes.html anssik