13:51:13 <RRSAgent> RRSAgent has joined #webmachinelearning
13:51:13 <RRSAgent> logging to https://www.w3.org/2022/09/08-webmachinelearning-irc
13:51:15 <Zakim> RRSAgent, make logs Public
13:51:16 <Zakim> please title this meeting ("meeting: ..."), anssik
13:51:20 <anssik> Meeting: WebML WG Teleconference – 8 September 2022
13:51:26 <anssik> Chair: Anssi
13:51:38 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-09-08-wg-agenda.md
13:51:41 <anssik> Scribe: Anssi
13:52:23 <anssik> scribeNick: anssik
13:52:27 <anssik> scribe+ dom
13:52:36 <anssik> ghurlbot, this is webmachinelearning/webnn
13:52:36 <ghurlbot> anssik, OK
13:52:44 <anssik> Present+ Anssi_Kostiainen
13:53:56 <anssik> RRSAgent, draft minutes
13:53:56 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
13:56:00 <anssik> RRSAgent, make logs Public
13:56:05 <anssik> RRSAgent, draft minutes
13:56:06 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
13:59:58 <ningxin_hu> ningxin_hu has joined #webmachinelearning
14:00:06 <anssik> Present+ Rafael_Cintron
14:00:11 <anssik> Present+ Ningxin_Hu
14:00:22 <bruce_dai> bruce_dai has joined #webmachinelearning
14:00:28 <anssik> Present+ Humera_Noor
14:00:54 <anssik> Present+ Bruce_Dai
14:01:03 <anssik> Present+ Dominique_Hazael-Massieux
14:01:36 <anssik> Present+ Chai_Chaoweeraprasit
14:03:22 <anssik> RRSAgent, draft minutes
14:03:22 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
14:03:41 <anssik> Topic: WebNN API Candidate Recommendation open issues
14:03:49 <anssik> -> Current CR issues https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Acr
14:04:10 <anssik> Subtopic: Support asynchronous context creation
14:04:18 <RafaelCintron> RafaelCintron has joined #webmachinelearning
14:04:31 <anssik> anssik: issue #272 has the latest discussion
14:04:32 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/272 -> Issue 272 Support asynchronous context creation (huningxin) cr
14:04:40 <chai> chai has joined #webmachinelearning
14:04:49 <anssik> ... two PRs for two design alternatives
14:05:13 <anssik> ... PR #274 is using the x() + xSync() pattern (Sync postfix)
14:05:14 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/274 -> Pull Request 274 Support async context creation and use sync postfix (huningxin)
14:05:40 <anssik> ... PR #285 is using the xAsync() + x() pattern (Async postfix)
14:05:40 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/285 -> Pull Request 285 Introduce MLContext.createContextAsync (huningxin)
14:06:06 <anssik> ... good discussion, no clear consensus because naming is hard!w
14:06:31 <anssik> ... based on feedback from Domenic it appears #274 aligns with the (ideal) web platform convention
14:06:49 <anssik> ... based on our investigation, #285 aligns with the emerging WebGPU API convention
14:07:19 <anssik> ... a new data point from Domenic was that a worker-only sync API is an exception
14:07:36 <anssik> ... I know this issue needs a compromise, so everyone should think: "can I live with the chosen design"
14:07:52 <anssik> q?
14:08:41 <anssik> q?
14:08:51 <dom> q+
14:08:54 <anssik> ack dom
14:09:55 <anssik> dom: I guess Sync APIs should be exceptions, so Sync version is used by code generators so postfixing with Sync, longer complex name for those consumers
14:10:21 <anssik> q?
14:10:42 <dom> anssi: that aligns with Domenic's feedback
14:11:05 <anssik> q?
14:11:11 <RafaelCintron> q+
14:11:16 <anssik> ack RafaelCintron
14:11:28 <dom> ... and that sync-only worker apis are expected to disappear given evolution of JS itself
14:11:39 <dom> s/JS itself/JS-WASM integration
14:12:26 <dom> Rafael: my feedback has been that a CPU-blocking op should return a promise; an operation that doesn't block on the CPU (e.g. because they run on the GPU) should return immediately
14:12:43 <dom> ... the CPU & GPU timelines are completely different
14:13:46 <chai> q+
14:13:57 <dom> ... our API should reflect where the job will be running and behave sync or async accordingly
14:14:13 <anssik> q?
14:14:16 <anssik> ack chai
14:15:13 <dom> chai: I agree with Rafael on the pattern, but since we're providing an abstraction over both the GPU & CPU timelines, I don't think we can adjust it that way
14:15:27 <dom> ... otherwise we would have to expose different APIs for CPU / GPU, which I think would be more confusing
14:15:53 <dom> ... the issue at hand is about naming here
14:16:45 <dom> ... since there is no consistency, looking at the WebGPU prior art of using -Async, where WebGL has been using -Sync
14:17:03 <dom> ... in the fact of no perfect answer, I would follow the convention of WebGPU given that we no longer support WebGL
14:17:49 <dom> ... given our interop with WebGPU, consistency with it would be useful
14:17:57 <dom> q+ to suggest we ask TAG's input
14:18:11 <anssik> q?
14:18:59 <anssik> q?
14:20:11 <anssik> q?
14:20:12 <anssik> ack dom
14:20:12 <Zakim> dom, you wanted to suggest we ask TAG's input
14:20:36 <anssik> dom: looking at WebGPU API, it does not have the same split with sync API available in the worker context, but in main thread
14:20:45 <anssik> ... so sync API exception does not apply to WebGPU API
14:21:07 <anssik> ... I hear the argument with WebGPU consistency, but we're not actually consistent with main/worker thread exposure
14:21:21 <anssik> ... may be useful to ask advise from TAG on this specific issue
14:21:56 <anssik> ... no super critical, but that would help us get to a decision, think Sync postfix is cleanest path
14:22:22 <anssik> ... one of my hats in W3C is to make sure we provide a good developer experience, and think that pattern in a better fit -- not a make or break thing
14:23:20 <anssik> anssik: can we bring this up with TAG so that we could get an answer faster than for usual reviews?
14:23:45 <anssik> proposed RESOLUTION: Seek TAG recommendation on asynchronous context creation naming issue
14:24:48 <RafaelCintron> q+
14:25:00 <anssik> ack RafaelCintron
14:25:54 <anssik> s/in a better/is a better
14:26:18 <dom> Rafael: our abstraction design may create unneeded delays - hopefully our API avoids that risk
14:26:41 <anssik> q?
14:26:51 <anssik> RESOLUTION: Seek TAG recommendation on asynchronous context creation naming issue
14:27:13 <anssik> Subtopic: Define ULP (unit of least precision) tolerances for testing
14:27:24 <anssik> #265
14:27:24 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/265 -> Issue 265 Define ULP (unit of least precision) tolerances for Conformance testing of WebNN API (BruceDai) cr
14:27:34 <anssik> anssik: w-p-t PR has been updated: https://github.com/web-platform-tests/wpt/pull/34287
14:27:42 <anssik> ... two the most recent commits add the following changes:
14:27:54 <anssik> ... 1) add tests for concat / reshape / slice / split / squeeze / transpose with input tensors
14:28:01 <anssik> ... 2) Add tests for clamp and relu operations
14:28:15 <anssik> ... ask for Chai to review the proposed ULP tolerances
14:28:33 <chai> q+
14:28:46 <anssik> ack chai
14:29:20 <anssik> RRSAgent, draft minutes
14:29:20 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
14:29:48 <dom> chai: I've been working with my team for quite a while on this
14:30:04 <dom> ... we have internal values for DirectML, but they're not fit to use directly in a public way
14:30:28 <dom> ... we've been looking at how to make them leverageable in the WebNN context
14:31:00 <dom> ... we're very close - some of the ops won't be to rely ULP given the set of devices & the CPU/GPU abstraction
14:31:28 <dom> ... operators like add can use a 0-ULP tolerance, but pow or sin/cosin are very complex and implementation-based
14:31:36 <dom> ... a ULP test won't be reliable for these
14:31:49 <dom> ... we've been tabulizing these recommendations and we're close to share them out
14:32:06 <dom> ... it won't be short answer
14:33:38 <dom> ... one of our senior engineer in our team has been working on this and may be providing the input on the issue
14:33:45 <bruce_dai> q+
14:33:58 <anssik> ack bruce_dai
14:34:02 <dom> anssi: we'll be happy to invite him on a call to discuss this
14:34:20 <dom> bruce_dai: thanks chai - looking forward to the input
14:34:31 <dom> ... the ULP test assertion has landed in WPT
14:36:20 <dom> dom: we can land WPT tests through our own internal review when we don't touch files outside of webnn
14:36:38 <anssik> q?
14:36:54 <anssik> Subtopic: Add method steps and normative algorithms to operations
14:37:01 <anssik> anssik: issues #210 and #211
14:37:02 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/211 -> Issue 211 Define algorithms for dictionaries with lists as default values (anssiko)
14:37:02 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/210 -> Issue 210 Add method steps to operations (anssiko) cr
14:37:14 <anssik> ... Zoltan volunteered to help with these issues, he sent regrets for today, so we'll get back to this on our next meeting
14:37:22 <anssik> Subtopic: Support for int8 quantized models
14:37:28 <anssik> #128
14:37:29 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/128 -> Issue 128 WebNN should support int8 quantized models (wchao1115) cr
14:37:45 <anssik> anssik: on our previous call we heard support from Rafael/Msft and Jonathan/Google not opposed
14:37:51 <anssik> ... Ningxin shared implementation options:
14:38:12 <anssik> ... - quantize/dequantize operations that frameworks could use within graphs
14:38:18 <anssik> ... - introduce this as an operand descriptor
14:38:23 <anssik> ... thoughts on these options?
14:39:15 <anssik> ... - also, another approach to introduce quantized ops, such as quantizedConv etc.
14:39:32 <dom> s/... -/Ningxin: /
14:39:37 <anssik> ... the WG should choose the quantization schema we need to survey the existing native APIs
14:40:26 <anssik> ... Chai we recall the spec PR for this was on your list?
14:40:51 <dom> chai: support quantized int8 is important for NPU, even if they're not yet in scope
14:41:19 <ningxin_hu> q+
14:41:25 <anssik> q?
14:41:26 <dom> anssi: do we still think we can have it included in the spec by EOY/CR?
14:41:30 <anssik> ack ningxin_hu
14:42:18 <dom> ningxin_hu: it's important for NPU, but int8 quantization is optimized in CPU & GPU for some hardware configuration
14:42:24 <dom> ... that would lead to performance boost there
14:42:28 <dom> q+
14:42:28 <anssik> s/Chai we/Chai I
14:42:31 <anssik> q?
14:42:33 <anssik> ack dom
14:42:58 <anssik> dom: I'm hearing there's value to get this feature done, but not clear who is going to make a proposal
14:43:10 <anssik> ... it would need to land in the upcoming few months
14:43:36 <anssik> ... should clarify whether we make this CR and who is driving it
14:43:52 <chai> q+
14:43:57 <anssik> ack chai
14:44:51 <dom> chai: is this asking whether to hold our CR transition until we add int8 support?
14:45:06 <dom> anssi: essentially
14:45:25 <dom> chai: if the timeline is set re shipping, we need to look at what we have
14:45:50 <dom> ... and figure what we can live without
14:46:45 <dom> anssi: our work is driven by use cases from which we've derived our features for performant operations on current or soon-to-be-current hardware
14:46:57 <dom> ... that said, our CR is not the end
14:46:59 <dom> q+
14:47:24 <anssik> -> Current CR issues https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Acr
14:48:09 <anssik> ack dom
14:48:50 <anssik> dom: is int8 quant nice to have or must have for CR?
14:50:05 <anssik> q?
14:50:17 <ningxin_hu> q+
14:50:24 <anssik> ack ningxin_hu
14:51:51 <dom> ningxin_hu: it looks like int8 is a feature addition compared to algorithm steps is a bug fix
14:52:01 <chai> q+
14:52:30 <dom> ... algo steps looks quite a bit of editorial work given the many operations we have to write up
14:52:37 <anssik> ack chai
14:53:05 <dom> chai: I think it looks quite doable to keep quantized ops, possibly as a new device type
14:53:22 <dom> ... to make sure we wouldn't break the API by adding it later
14:53:35 <dom> ... esp with the idea of adding the NPU device type
14:53:41 <dom> q+
14:54:00 <anssik> ack dom
14:54:48 <anssik> dom: I think it is good idea to evaluate whether our API can work with new device types such as NPUs, not sure if NPUs should be in CR release scope due to required implementation experience
14:55:24 <anssik> ... quantization for existing devices is fine, if this effort only makes sense in the context of NPU support I'm less sure
14:56:53 <anssik> anssi: we can identify features as at risk to allow exiting CR in time if implementation feedback (for e.g. NPU) is not received
14:57:21 <anssik> anssik: evalution now is good, dom?
14:57:31 <anssik> dom: correct
14:57:44 <anssik> q?
14:57:54 <anssik> Topic: Use cases for future work
14:58:03 <anssik> Subtopic: Content filtering
14:58:12 <anssik> anssik: Humera submitted a PR #253 that was presented to the group that recommended this as a future work item
14:58:13 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/253 -> Pull Request 253 Add "Ethical Content Filtering" use case to WebNN specs (humeranoor)
14:58:18 <anssik> -> Future work proposal: Content Filtering https://github.com/webmachinelearning/proposals/issues/4
14:58:48 <anssik> ... to keep discussion in one place, proposal to close the PR
14:59:06 <anssik> ... to implement this use case, the web community needs to agree on web extensions and improvements to other APIs such as webRequest and declarativeNetRequest
14:59:24 <anssik> ... thanks Humera!
15:00:14 <anssik> Humera: I don't mind closing the PR now
15:00:24 <Humera_eyeo> https://webmachinelearning.github.io/webnn/#usecases-application
15:01:23 <anssik> Humera: use cases drive WebNN API definition and many nice applications in this list, we found content filtering use case was missing, and if this use case is in the list, it will help convince other web extension developers to change their APIs
15:02:16 <anssik> dom: there's a very active CG for web extensions
15:02:41 <anssik> https://www.w3.org/community/webextensions/
15:02:42 <anssik> https://github.com/w3c/webextensions
15:03:19 <anssik> Subtopic: Performance adaptation
15:03:24 <anssik> #207
15:03:24 <ghurlbot> https://github.com/webmachinelearning/webnn/issues/207 -> Pull Request 207 Update "Performance Adaptation" use case (spshin3)
15:03:33 <anssik> anssik: this PR has been open for over a year now and we have had an extensive discussion on it
15:03:45 <anssik> ... based on review comments, it is suggested this use case should be submitted as a future work proposal https://github.com/webmachinelearning/proposals
15:03:51 <anssik> ... and we should this PR
15:03:55 <anssik> ... any concerns with that?
15:04:09 <anssik> [no concerns heard to close PR]
15:04:22 <anssik> RRSAgent, draft minutes
15:04:22 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
15:04:41 <anssik> RRSAgent, draft minutes
15:04:41 <RRSAgent> I have made the request to generate https://www.w3.org/2022/09/08-webmachinelearning-minutes.html anssik
17:03:27 <Zakim> Zakim has left #webmachinelearning