13:51:13 RRSAgent has joined #webmachinelearning 13:51:13 logging to https://www.w3.org/2022/10/20-webmachinelearning-irc 13:51:15 RRSAgent, make logs Public 13:51:19 please title this meeting ("meeting: ..."), anssik 13:51:19 Meeting: WebML WG Teleconference – 20 October 2022 13:51:20 Chair: Anssi 13:51:24 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-10-20-wg-agenda.md 13:51:30 Scribe: Anssi 13:51:34 scribeNick: anssik 13:55:34 Present+ Anssi_Kostiainen 13:55:38 Regrets+ Dominique_Hazael-Massieux 13:55:46 RRSAgent, draft minutes 13:55:46 I have made the request to generate https://www.w3.org/2022/10/20-webmachinelearning-minutes.html anssik 14:00:08 bruce_dai has joined #webmachinelearning 14:00:30 ningxin_hu has joined #webmachinelearning 14:00:48 Present+ Zoltan_Kis 14:00:57 Present+ Bruce_Dai 14:01:03 Present+ Ningxin_Hu 14:03:35 zkis has joined #webmachinelearning 14:03:39 Present+ Chai_Chaoweeraprasit 14:03:45 Present+ Rafael_Cintron 14:03:46 RafaelCintron has joined #webmachinelearning 14:04:29 chai has joined #webmachinelearning 14:05:34 Topic: WebML WG Charter 2023-2025 under development 14:06:22 anssik: Web Machine Learning Working Group Charter for 2023-2025 is now under development. 14:07:16 ... Please review the draft PR and open issues, provide your comments and open new issues as appropriate to help shape the WG's technical scope. 14:07:21 -> Announcement https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2022Oct/0002.html 14:07:27 -> Charter PR https://github.com/w3c/machine-learning-charter/pull/19 14:07:36 -> Charter open issues https://github.com/w3c/machine-learning-charter/issues 14:08:05 anssik: in the charter PR I included the expected timeline: 14:08:12 ... - Q4 '22: Charter development 14:08:25 ... - Q1 '23: W3C Advisory Committee review 14:08:50 ... - Q2' 23: Charter approved 14:09:13 anssik: I propose we quickly touch each of the issue that has been recorded based on the feedback from the WG 14:09:29 ... if you have immediate feedback or comments, feel free to queue yourself, otherwise please provide your feedback in the GH issues 14:09:37 q+ 14:09:38 ... we expect to have a good draft charter ready EOY 2022 14:09:41 ack RafaelCintron 14:09:56 RafaelCintron: how the charter is changing from the last time? 14:10:26 anssik: up to the WG 14:11:21 ... we can also ask for an extension 14:11:30 RafaelCintron: what is the proposal then, keep going or change? 14:12:50 RafaelCintron: I'm happy with our current charter, but happy to review any adjustments too 14:13:59 q? 14:14:29 Subtopic: Features deferred to WebNN v2 14:15:04 -> Features deferred to WebNN v2 https://github.com/w3c/machine-learning-charter/issues/25 14:15:41 https://github.com/webmachinelearning/webnn/labels/v2 14:15:53 anssik: "WebNN v2" is a construct that refers to the WebNN API spec post initial Candidate Recommendation 14:16:11 ... v1 is in itself is expected to be a useful API 14:16:37 ... the WG has labeled a few issues as "v2", so please check out those and let us know if you have other suggestion 14:17:18 https://github.com/webmachinelearning/webnn/issues/128 14:18:18 Subtopic: Dedicated ML hardware accelerators: NPU, VPU, xPU 14:18:28 -> Dedicated ML hardware accelerators: NPU, VPU, xPU https://github.com/w3c/machine-learning-charter/issues/24 14:18:43 anssik: The initial version of WebNN specifies two device types, "cpu" and "gpu". 14:19:01 ... However, the API is extensible with new device types and in our discussion support for NPU, VPU, or XPU has come up as a new "v2" feature. 14:19:26 ... The initial charter refers to "dedicated ML hardware accelerators" in its Motivation and Background, but if this is important we could be more explicit regarding NPU/VPU/XPU device type support. 14:19:55 q? 14:20:14 q+ 14:20:19 ack chai 14:21:55 chai: I think NPU is quite important especially for mobile scenarios 14:22:09 ... if we can make it part of the charter, it is quite important 14:23:23 q? 14:24:35 anssik: is a unified hardware-agnostic WebNN API the WG's primary goal? 14:24:41 chai: I think so 14:25:27 q+ 14:25:30 ack ningxin_hu 14:25:59 ningxin_hu: talked with Chai about this unified agnostic behaviour 14:26:27 q+ 14:26:35 ... some hardware platforms may not support e.g. some data types, this should be defined by the WebNN spec that whether it allows detecting features so web apps can adapt to the hardware differences 14:26:46 ... also backwards compatibility, versioning considerations 14:26:58 q+ 14:27:01 ... when we go to WebNN v2 how we deal with this versioning and feature detection 14:27:17 q? 14:27:32 ack zkis 14:28:31 zkis: to reinforce Chai, depending on the underlying platform, we can do op without an error, but it is dynamic, cannot know it a priori 14:28:36 ack chai 14:29:09 chai: re fallback, there are two use cases, WebNN is a backend, fw is on top 14:29:29 ... 1st case fw can handle fallback, e.g. CPU fallback 14:30:10 ... 2nd use case, NPU may get more popular, can only support a subset of ops 14:30:33 ... the fallback needs to happen below the WebNN API 14:30:47 ... in the 1st use case the fallback happens above WebNN in the fw 14:31:12 ... example, given "auto" device type, then the fallback happens underneath the WebNN implementation 14:31:36 ... the second use case is important to v2 I think, especially when the app says "I don't care, handle this for me" 14:31:48 ... e.g. app does not want to deal with the error codes 14:32:07 ... if we make explicit what errors make WebNN fail we have a possible fingerprint issue 14:33:02 q? 14:34:26 q+ 14:34:32 ack zkis 14:36:54 q? 14:37:02 Subtopic: Set of ops supported must be more comprehensive 14:37:07 -> Set of ops supported must be more comprehensive https://github.com/w3c/machine-learning-charter/issues/23 14:37:14 anssik: Chai shared that external partners are looking for a more comprehensive set of ops 14:37:30 ... The current charter Scope enumerates a few common ones: "convolution, pooling, softmax, normalization, fully connected, activation, recurrent neural network (RNN) and long short-term memory (LSTM)". 14:37:52 ... This is not meant to be an all inclusive list and does give the WG ability to adapt to the changes in this landscape. 14:38:10 ... At minimum, we should review the bullets in the Scope section, and see whether to explicitly mention some of the more recent work such as transformers. 14:38:29 ... We want to give enough detail to give good direction without constraining the WG too much. The list of ops mentioned in the charter would be open-ended. 14:39:31 chai: transformer is a huge class of models 14:40:03 ... this is a big class of emerging models, but not very information 14:40:18 ... natural language processing is a friendlier to the audience 14:40:30 ... NLP represents current transformer models 14:41:04 ... in future transformers may become less popular when the next hotter one comes around, similarly to LSTMs in the past 14:41:39 q? 14:41:54 q+ 14:41:57 ack ningxin_hu 14:42:32 ningxin_hu: I want to clarify what Chai said, do you mean the charter should focus more on usages e.g. NLP or computer vision? 14:42:38 ... usage can change from time to time 14:42:48 ... we use different ML techniques to address these usages 14:43:17 ... architectures change, do you suggest we focus more on usages? 14:44:48 Chai: correct, given the popularity of these more recent models 14:45:24 q? 14:45:32 Subtopic: Level of abstraction for neural net operations 14:45:39 -> Level of abstraction for neural net operations https://github.com/w3c/machine-learning-charter/issues/22 14:45:54 anssik: WebNN explainer has a nice section that explains the rationale for the chosen level of abstraction for the neural network operations in WebNN API. 14:46:07 ... Chai proposed we could integrate some of this explainer text into the next charter to provide more context on the level of abstraction. This could fit into the Scope section. 14:46:49 ... I recall past discussion around this topic, for example Google was interested in XLA (Accelerated Linear Algebra) domain-specific compiler compatibility. XLA project seems to be moving to an open governance mode and is being decoupled from the TensorFlow project. 14:47:10 ... I'd welcome someone from Google to talk about their plans and expectations with XLA and its input language HLO IR (High Level Operations), and how they see it being part of the WebNN implementation story. 14:47:23 q? 14:47:33 ack zkis 14:47:34 q- 14:49:00 q+ 14:49:11 q+ 14:49:15 ack ningxin_hu 14:49:34 ningxin_hu: XLA moved to open governance, OpenXLA 14:50:07 ... Google previously proposed that, we investigated that with Chai, mapping ops to ONNX and XLA, gap was not so big to me 14:50:23 ... I'd look forward to a concrete proposal from that community to understand the mapping to that abstraction and gap 14:50:35 ... there was another proposal TOSA (Tensor Op Set Arch) 14:51:11 ... we also investigated that, my question probably cannot be answered right now, but I'd like to understand whether we should follow up closely with one of these 14:51:40 q? 14:51:51 q+ 14:51:57 ack chai 14:52:24 chai: just quickly, I'm aware of OpenXLA when they started, not sure what they intend to do with it 14:52:38 ... it seems to be split from TensorFlow project 14:52:50 q- 14:52:59 q+ 14:53:02 ... it is good for us to point to them they should strive for compat with WebNN 14:53:04 q? 14:53:12 ack ningxin_hu 14:53:43 ningxin_hu: WebNN positions itself as backend framework 14:54:18 ... another issue is graph compiler, we should make it clear what is WebNN position in this stack regarding DL compiler 14:54:49 ... is compiler a implementation backend, or use WebNN for codegen, or complementary to WebNN 14:55:24 ... Google raised an issue for MLIR to this WG in the past, that questions did not last for long, but there are some DL compilers that are actively developed 14:56:32 q? 14:56:53 Subtopic: WebRTC coordination 14:56:57 -> WebRTC coordination https://github.com/w3c/machine-learning-charter/issues/21 14:57:13 anssik: We added an Integration with real-time video processing use case based on learnings from our experimentation into WebNN API spec 14:57:25 ... For the next charter, we could be more explicit and confident in Coordination re WebNN and WebRTC 14:57:32 ... I made a suggestion in the issue and Dom +1'd it, so I'm thinking of tweaking the WebRTC coordination accordingly. 14:57:37 ... any further suggestions for perhaps even more explicit text for our WebRTC integration interests? 14:57:57 q? 14:58:06 Subtopic: WebGPU interoperability 14:58:12 -> WebGPU interoperability https://github.com/w3c/machine-learning-charter/issues/20 14:58:41 anssik: We discussed WebGPU interoperability expectations on our 6 October 2022 call and concluded working with WebGPU contributors is important for the success of the WebNN spec. I'd want us to revise the charter language around WebNN-WebGPU interoperability expectations accordingly. 14:58:56 ... The initial charter mentions WebGPU in the context of Out of Scope and Coordination 14:59:02 ... I think this needs to be revised to reflect our evolved thinking. For example: 14:59:10 ... "to avoid overlap with existing work, generic primitives used by traditional machine learning algorithms such as base linear algebra operations are out of scope. The WebGL and WebGPU shaders and WebAssembly SIMD are expected to address these requirements, see the Coordination section for details." 14:59:45 q? 15:00:04 q+ 15:00:06 ack ningxin_hu 15:00:51 ningxin_hu: current charter address usage of custom ops, early WebNN issue re custom ops with WebNN and the solution to say custom ops can be implemented with more generic APIs, Wasm SIMD etc. 15:01:18 ... Raphael mentioned some use cases e.g. super resolution, require WebNN to interact with WebGPU with resource and buffer sharing 15:01:35 ... this is not mentioned in the current charter, if this usage is important, propose to make this more explicit 15:02:45 RafaelCintron: WebGPU interop is critical and this should not compromise the perf of the API, details subject to discussion 15:03:16 RRSAgent, draft minutes 15:03:16 I have made the request to generate https://www.w3.org/2022/10/20-webmachinelearning-minutes.html anssik 16:33:34 zkis_ has joined #webmachinelearning 16:59:48 dom has joined #webmachinelearning 17:27:44 Zakim has left #webmachinelearning