13:53:24 RRSAgent has joined #webmachinelearning 13:53:24 logging to https://www.w3.org/2022/05/19-webmachinelearning-irc 13:53:26 RRSAgent, make logs Public 13:53:27 please title this meeting ("meeting: ..."), anssik 13:53:31 Meeting: WebML WG Teleconference – 19 May 2022 13:53:37 Chair: Anssi 13:53:42 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-05-19-wg-agenda.md 13:53:47 Scribe: Anssi 13:53:52 scribeNick: anssik 13:53:57 scribe+ dom 13:54:13 ghurlbot, this is webmachinelearning/webnn 13:54:13 anssik, OK 13:54:27 Present+ Anssi_Kostiainen 13:54:37 RRSAgent, draft minutes 13:54:37 I have made the request to generate https://www.w3.org/2022/05/19-webmachinelearning-minutes.html anssik 14:02:51 Present+ Rafael_Cintron, Ningxin_Hu, Dominique_Hazael-Massieux, Geun-Hyung_Kim 14:03:06 Geun-Hyung has joined #webmachinelearning 14:03:17 ningxin_hu_ has joined #webmachinelearning 14:03:24 Present+ 14:03:50 Topic: Announcements 14:04:00 present+ 14:04:03 Present+ Dominique_Hazael-Massieux 14:04:15 Zakim, who's here? 14:04:15 Present: Anssi_Kostiainen, Rafael_Cintron, Ningxin_Hu, Dominique_Hazael-Massieux, Geun-Hyung_Kim, ningxin_hu_, Geun-Hyung 14:04:18 On IRC I see ningxin_hu_, Geun-Hyung, RRSAgent, Zakim, AramZS, anssik, dom, ghurlbot, tomayac, iank_, `join_subline, gregwhitworth, sangwhan 14:04:19 Present+ Rafael_Cintron 14:04:30 Present+ Chai_Chaoweeraprasit 14:04:50 q? 14:05:03 chai has joined #webmachinelearning 14:05:12 anssik: the 1st WebNN API Chromium patch landed! 14:05:23 ... congrats to everyone who helped get through the review process 14:05:46 ... collaboration across several companies working together 14:05:47 congratulation! 14:06:18 https://chromium-review.googlesource.com/c/chromium/src/+/3583762 14:06:35 RafaelCintron has joined #webmachinelearning 14:06:42 ningxin_hu: the patch was submitted by Junwei 14:07:00 Jonathan has joined #webmachinelearning 14:07:01 ... we had started the intent to prototype process together with Rafael and Chai 14:07:12 ... This is the first patch with Blink-side IDL definition with MLGraphBuilder 14:07:52 ... it's the very minimal initial set to try the blink interfaces 14:08:06 ... it's aligned with what Honglin has done for the Model Loader API 14:08:16 ... they're making good progress in the ChromeOS team 14:08:32 ... we've discussed aligning interfaces, e.g. the MLContext interface would be shared between the two APIs 14:09:11 ... Thanks to Honglin for his review and others like Jonathan for their help to go through this 14:09:14 RRSAgent, draft minutes 14:09:14 I have made the request to generate https://www.w3.org/2022/05/19-webmachinelearning-minutes.html anssik 14:09:25 ... Junwei sent a second patch that adds more methods of the MLGraphBuilder 14:09:37 ... targeting the 1st wave ops identified in the spec 14:10:32 ... we'll be adding more, and figure out the next steps of implementations e.g. integration with the native backends 14:10:35 ... as explored in webnn-native 14:10:44 -> webnn-native open-source project https://github.com/webmachinelearning/webnn-native/ 14:10:50 ... we'll need to figure out which backends can be brought to Chromium 14:11:06 ... we hope to bring webnn and make it available for testing soon 14:11:19 Congrats to everyone involved indeed! 14:11:51 Topic: WebNN API Candidate Recommendation - release vs. post-release scope 14:12:17 anssi: we're marching towards the CR spec milestone 14:12:27 ... to get there, we need to figure out the technical scope of the spec 14:12:52 ... please look at the current issues for the spec and triage them for release (things to fix by CR), and post-release (things to address after CR) 14:12:56 -> Current CR issues https://github.com/webmachinelearning/webnn/issues?q=is%3Aissue+is%3Aopen+label%3Acr 14:13:52 anssi: CR indicates a spec is considered complete and fit for purpose 14:14:14 the 2nd CL, feel free to help review: https://chromium-review.googlesource.com/c/chromium/src/+/3650912 14:14:22 ... no further text changes are expected beyond implementation feedback and testing, a spec that can be meaningfully implemented as is 14:15:15 ... please note issues that you think need to be tracked for our CR milestone 14:15:21 ... we're committed to hit during this year 14:15:29 q+ 14:15:34 ack chai 14:15:56 chai: do we believe the list of issues is complete? does it need a full review? 14:16:06 anssik: it's not finalized, it's our current best understanding 14:16:34 ... there may be additional issues - the WG participants should identify those 14:16:55 chai: we have quite a few operators defined, but that list can never be completed 14:17:16 ... we probably have between 30-50% of what complete would look like 14:17:29 ... what criteria do we believe the list of operators for v1 need to fulfill? 14:17:53 anssik: one approach could be to look at the use cases we identified for spec - do the current opset satisfy the scope of these use case? how is the match? 14:18:10 ... what you said is a key point - the number of ops is growing, but we're not going to 100% with this release 14:18:33 ... we need to explain our story for this particular set, and how we might grow it in the future 14:18:47 ... e.g. similar to how it may have been done for ONNX 14:18:56 ... it may be appropriate to express this in the spec itself 14:19:10 chai: it would be useful to ask for the group what kind of models we want to support for V1 14:19:26 ... we make a good job for CNN; do we need to support Transformers for V1? 14:19:41 ... this may lead to adding more operators 14:19:56 ... Transformers is an example of a popular model kind, but there may be others 14:19:57 -> The first-wave models https://github.com/webmachinelearning/webnn/blob/main/op_compatibility/first_wave_models.md 14:20:09 anssik: the first wave models document has been guiding the initial work 14:20:25 ... one task might be to make sure that these first wave models reflects the current state of reality 14:21:13 ... clearly the opset target is a key consideration for our CR milestone 14:21:35 q? 14:22:05 Topic: Support asynchronous graph compilation 14:22:18 #263 14:22:19 -> Issue 263 Support asynchronous graph compilation (wchao1115) https://github.com/webmachinelearning/webnn/issues/263 14:22:45 #266 14:22:46 -> Pull Request 266 Add buildAsync method for async graph compilation (huningxin) https://github.com/webmachinelearning/webnn/issues/266 14:23:22 anssik: the PR LGTM - we could discuss it now or on github 14:23:42 i'll take a look at PR 266 14:23:53 ningxin_hu: mostly straightforward PR based on #257 14:23:54 -> Pull Request 257 [closed] Context-based graph execution methods for different threading models. (wchao1115) https://github.com/webmachinelearning/webnn/issues/257 14:23:59 RRSAgent, draft minutes 14:23:59 I have made the request to generate https://www.w3.org/2022/05/19-webmachinelearning-minutes.html anssik 14:24:30 ... we have computeAsync, Junwei requested an async build method for main thread usage 14:25:10 ... this is needed to ensure a graph is ready for efficient execution 14:26:19 ... with this PR, when the promise resolves, the graph is ready for execution 14:26:20 q? 14:26:42 Topic: CommandBuffer usage clarification 14:26:59 #264 14:26:59 -> Issue 264 CommandBuffer usage clarification: internal, external, both? (bbernhar) https://github.com/webmachinelearning/webnn/issues/264 14:27:16 "I was seeking for clarification on the usage of MLCommandBuffer: will it be internal-only (read/write by WebNN), external-only (read-only by WebNN, read/write by WebGPU) or both (read/write both WebNN and WebGPU)." 14:27:55 -> WebNN API - WebGPU Interoperability section https://www.w3.org/TR/webnn/#api-mlcontext-webgpu-interop 14:28:26 q? 14:28:51 sorry i'll have a hard stop this morning at 7:40 14:29:11 q+ 14:29:20 anssik: one of the underlying issue is if the WebGPU API changes under our feet, this may create issues for us 14:29:21 q? 14:29:23 ack chai 14:29:42 chai: this is one of the last topics for the async discussion 14:29:52 ... we want to separate the interface from the implementation 14:30:19 ... we don't need to reinvent a new type for a commandbuffer - there aren't a lot of benefits to reinvent the WebGPU command buffer 14:30:53 ... Brian's feedback is on the implementation side - the two new APIs need to agree and allow them working together under the hood 14:31:11 ... In terms of API, I think it doesn't really matter whether it's ML- or GPUCommandBuffer 14:31:48 ... from the implementation perspective, we need to work with the Chromium Dawn project to make sure that what WebNN produces can be used by WebGPU 14:31:50 q+ 14:32:08 q? 14:32:52 q+ 14:32:58 ack dom 14:33:32 dom: thanks Chai, I want to make sure this implementation concern isn't Chromium specific so that decision in this space do not make it harder for other engines to adopt WebNN 14:34:07 ... nothing suggests that would be the case, but want to make sure we do not optimize for Chromium implementation, but enable other implementations with different architectures to perform equally well 14:34:11 q? 14:34:24 chai: +1 14:35:33 ... from my experience, the interop between WebGPU & WebNN is helped a bit by the API, but the key is at the implementation level 14:35:44 ... the key thing is to make sure the API doesn't prevent from the right thing to happen 14:35:55 ... currently, GPUCommandBuffer is obviously not WebNN-aware 14:36:17 q? 14:36:20 ack RafaelCintron 14:36:21 ... the question is whether the implementation of GPUCommandBuffer can be amended to handle WebNN needs 14:36:33 RafaelCintron: +1 14:37:02 ... the bottom line is that we need implementation experience to give us clarity on the API shape 14:38:29 q? 14:38:32 anssik: please bring that input to the issue as well (I'll post the minutes there already) 14:38:49 Topic: WebNN API Ethical Considerations integration 14:39:10 ghurlbot, this is ethical-webmachinelearning 14:39:11 dom, OK 14:39:32 -> Ethical Principles for Web Machine Learning https://webmachinelearning.github.io/ethical-webmachinelearning/ 14:39:37 anssik: we're expected to integrate mitigations in the WebNN spec informed by the work done on the ethical principles for machine learning 14:39:54 ... we've received substantial contributions from people outside the group 14:39:55 #20 14:39:56 -> Pull Request 20 Incorporate feedback from the Ethical ML workshops (anssiko) https://github.com/webmachinelearning/ethical-webmachinelearning/issues/20 14:40:23 -> WG's commitment to ethics: https://www.w3.org/2021/04/web-machine-learning-charter.html#ig-other-deliverables 14:40:46 ... this is inline with our charter commitment 14:41:25 q? 14:41:29 ... we will need to start integrating mitigations in the WebNN spec - it might be hard to translate from abstract ethical issues to mitigations, but there may be areas to provide implementors guidance 14:42:12 dom: I think once I get to review the PR we should look at publishing first public W3C Note of the Ethical Principles doc 14:42:30 ... deriving mitigations from principles will be another task, substantial work item 14:43:43 anssik: at the minimum I expect we will surface informative indications in WebNN; maybe some normative mitigations, unclear yet 14:44:02 ... this is a first in a W3C WG 14:44:11 q? 14:44:21 Topic: WebNN API Privacy Review refresh 14:45:17 anssik: an earlier version of the spec got reviewed by the Privacy IG 14:45:32 ... and committed to submit another review on potentially privacy-impacting work 14:46:08 ... what has changed since the initial review that would have privacy implications? 14:46:18 ... we have at least one open issue on device selection 14:46:20 ghurlbot, this is webnn 14:46:21 dom, OK 14:46:21 #169 14:46:21 -> Issue 169 Device selection with MLDevicePreference and MLPowerPreference (anssiko) https://github.com/webmachinelearning/webnn/issues/169 14:46:45 q? 14:47:12 dom: I think the safest thing would be to do a diff and see what has changed send that to PING 14:47:26 ... or review the PRs that got merged since last review 14:49:20 Topic: WebGPU and WebGL WG review 14:50:09 anssik: we have dependencies with WebGL & WebGPU 14:50:20 ... we haven't interacted much with the Khronos WebGL WG yet 14:50:24 gpuweb/gpuweb#2500 14:50:24 -> Issue 2500 Investigation: how WebNN / WebGPU interop could be happening (huningxin) https://github.com/gpuweb/gpuweb/issues/2500 14:51:06 ... for WebGPU, we have an ongoing conversation and have a plan to share with them our latest design 14:51:33 ... do we need to coordinate with WebGL in some manner? 14:51:45 q? 14:52:09 RafaelCintron: if we have WebGL datatypes in WebNN, yes - the same way if WebNN datatypes were being used elsewhere 14:52:51 ... once WebGPU ships, I expect WebGL will move to a maintenance mode; if interop with WebGL is too costly, we could also consider not interop-ing with it esp once WebGPU ships 14:53:16 anssik: sounds like a CR scoping consideration 14:53:17 q+ 14:53:24 ack dom 14:53:40 dom: Rafael, do you expect there are dragons in WebGL interop? 14:54:30 RafaelCintron: speaking for Chrome on Windows, WebGL implementation is built in DirectX 11 which will create lots of GPU copies and surprising performance degradation 14:54:47 dom: the dragons are in terms of reality of implementation than dragon in the spec interop? 14:56:20 ... do we see value in providing WebML interop? Or is it because WebGPU was uncertain when we started WebNN? 14:56:37 RafaelCintron: I would favor dropping WebGL interop given the reality of implementations 14:56:49 ... WebGPU is the path forward 14:56:55 +1 to drop WebGL interop 14:57:11 q+ 14:57:26 q? 14:57:33 ack ningxin_hu_ 14:57:52 ningxin_hu: +1 to opening an issue on this; I would also support dropping WebGL interop 14:58:20 ... when we started WebNN, the frameworks could only use WebGL to use GPU as a backend 14:58:26 ... that's the historical reason for having it 14:59:03 ... now that WebGPU is thet path forward and WebGL in maintenance, dropping WebGL will also help performance-wise for implementations 14:59:15 s/thet/the 14:59:54 ... my experience with the video processing sample #226, WebGL created issues for performance 14:59:55 -> Issue 226 Integration with real-time video processing (dontcallmedom)cr https://github.com/webmachinelearning/webnn/issues/226 15:00:09 ... which was confirmed by the tensorflow.js developer 15:00:13 RRSAgent, draft minutes 15:00:13 I have made the request to generate https://www.w3.org/2022/05/19-webmachinelearning-minutes.html anssik 15:00:32 ... this is a known issue in the WebGL chromium implementation with no plan to be fix it soon 15:00:54 q? 15:00:58 ... so +1 to dropping WebGL interop 15:01:09 Topic: Ethical Content Filtering use case 15:01:28 anssik: Humera provided more concrete use cases with technical requirements to support the proposed "Ethical Content Filtering" use case - please help review and see the PR comments 15:01:33 #253 15:01:33 -> Pull Request 253 Add "Ethical Content Filtering" use case to WebNN specs (humeranoor) https://github.com/webmachinelearning/webnn/issues/253 15:01:40 ... Use Case 1: Support for network level request filtering through WebNN's API 15:01:46 ... Use Case 2: Support for DOM access through WebNN's API 15:02:46 RRSAgent, draft minutes 15:02:47 I have made the request to generate https://www.w3.org/2022/05/19-webmachinelearning-minutes.html anssik 15:02:51 q?