14:01:07 RRSAgent has joined #webmachinelearning 14:01:07 logging to https://www.w3.org/2021/08/19-webmachinelearning-irc 14:01:09 RRSAgent, make logs Public 14:01:11 please title this meeting ("meeting: ..."), anssik 14:02:03 Meeting: WebML WG Teleconference – 19 Aug 2021 14:02:08 Chair: Anssi 14:02:12 Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2021-08-19-agenda.md 14:02:16 Scribe: Anssi 14:02:22 scribeNick: anssik 14:02:28 Present+ Anssi_Kostiainen 14:02:32 Regrets+ Dominique_Hazael-Massieux 14:02:38 Present+ Chai_Chaoweeraprasit 14:02:43 Present+ Rafael_Cintron 14:02:49 Present+ Ningxin_Hu 14:02:57 RRSAgent, draft minutes 14:02:57 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 14:03:52 anssik: Welcome back after the break! I hope those of you who had time off had a great time. I'm aware some of you are still on vacation. 14:03:55 Chai has joined #webmachinelearning 14:03:57 Geun-Hyung has joined #webmachinelearning 14:04:09 ping_yu has joined #webmachinelearning 14:04:12 present+ 14:04:18 ... I observed some great progress during the break and also substantial contributions from new contributors. Thank you! 14:04:50 Topic: Upcoming meetings 14:05:03 Subtopic: WG bi-weekly teleconference scheduling 14:05:14 anssik: We ran a poll to understand participants' preferences for our bi-weekly meeting 14:05:24 ... given our global participation, I appreciate folks flexibility as we obviously cannot find a time that is perfect for everyone, given the Earth is not flat 14:05:30 -> https://doodle.com/poll/4cdgqrdstew9yenu WebML WG teleconference scheduling poll 14:05:47 RafaelCintron has joined #webmachinelearning 14:05:59 anssik: looking at the results of the second poll we ran in August, the current Thu 14:00-15:00 UTC+0 slot received the most votes in support 14:06:28 ... in inclined to continue with the scheduling we have had in place and re-run the poll from time to time to understand if the situation has changed e.g. with new participants joining 14:07:13 ... I should note UTC time does not change with a change of seasons, so we have previously shifted to 15:00-16:00 UTC+0 when DST ends (~from end of Oct till end of Mar) which means 7am Pacific and 11pm Shanghai, given China Standard Time is observed all year 14:07:24 ... any concerns with that? 14:07:36 no problem to me 14:07:40 proposed RESOLUTION: Per scheduling poll results, stick with Thu 14:00-15:00 UTC+0 bi-weekly schedule for WebML WG teleconference during Daylight Saving Time, 15:00 UTC+0 when DST not observed 14:07:45 RESOLUTION: Per scheduling poll results, stick with Thu 14:00-15:00 UTC+0 bi-weekly schedule for WebML WG teleconference during Daylight Saving Time, 15:00 UTC+0 when DST not observed 14:07:53 Subtopic: WG meetings during W3C TPAC 2021 14:07:59 +1 14:08:09 -> https://www.w3.org/2021/10/TPAC/ W3C TPAC 2021 14:08:16 -> https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2021Aug/0000.html Dom's mail 14:08:49 anssik: W3C annual conference, TPAC, will happen in a virtual format, over the course of two weeks: October 18 to 29. Dates are split as: 14:09:04 ... 18 - 22 October unconference breakout sessions 14:09:12 ... 25 - 29 October WG/IG/BG/CG meetings and Joint Group Meetings 14:09:22 ... in our case, we'd meet over 1-2 days during 25-29 Oct. 14:09:35 ... quoting what Dom said in his email: 14:09:45 As a new Working Group, I believe it would be good for the Web Machine 14:09:45 Learning Working Group to organize such a meeting during the 2nd week. 14:09:45 In other words, such a meeting would be more an opportunity to reflect 14:09:45 on where we are, our next steps and explain the context of the group to 14:09:45 the broader community than it would be about solving specific technical 14:09:45 issues. 14:10:40 ... so the TPAC Group meetings are differently scoped from our usual bi-weekly meetings: 14:10:49 ... - look on the progress and goals of the Group as well as the deliverables 14:11:09 ... - look at related work (e.g in Community Groups) and what's new out there within the scope or related to the Group's mission; 14:11:26 ... - welcome new participants, understand their interests, get their questions/feedback on the Group, and potentially mentor them on how to contribute; 14:11:37 ... - welcome observers, understand their interests in the Group, and get them interested in joining the Group and helping; 14:12:01 anssik: at TPAC we discuss a more high-level view of the WG, its successes and challenges, goals, look at where we are to meet the charter goals 14:12:23 Present+ Ping_Yu 14:13:48 ... other topics: 14:14:08 ... - state of test suite and implementations 14:14:24 ... - past progress 14:14:41 ... - next 12 months expectations/hopes 14:15:00 ... - what contributions would be welcome from other/new participants 14:15:26 ... all this in addition to out usual business that is driven by issues/PRs 14:15:44 anssik: I'll check back with Dom on TPAC scheduling options and cross-group coordinations when he's back and we'll also ask your feedback. 14:15:56 ... any questions? 14:16:00 RRSAgent, draft minutes 14:16:00 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 14:16:25 Topic: WebNN API new features landed 14:16:39 anssik: I'd like us to discuss recently landed new features and any related design considerations behind them. 14:16:50 Subtopic: Fused activation 14:17:00 anssik: Fused action is related to two issues, #138 and #185 14:17:06 -> https://github.com/webmachinelearning/webnn/issues/138 Issue #138 14:17:10 -> https://github.com/webmachinelearning/webnn/issues/185 Issue #185 14:17:22 -> https://github.com/webmachinelearning/webnn/pull/188 PR #188 14:17:32 anssik: PR #188 by Chai addresses both these issues was reviewed and merged. 14:17:42 ... thanks to Chai, Ningxin, Rama, others who contributed. 14:18:07 ... the gist of these changes that generalize a way to allow an operator to be created before it is connected to the graph. 14:19:33 Chai: the only operators like this are activation function, you don't want to make this broadly for all ops, with this change we can pass activation function into things like convolution and BatchNorm 14:20:04 ... also addresses one TAG issue that flagged enum for a few activation functions 14:20:40 ... we solved this, but not in a fragile way that would have been to update the activation enum 14:21:02 ... also in some ops like GRU this would not make sense, and would make checking for errors hard 14:22:30 Subtopic: HardSwish op 14:22:35 -> https://github.com/webmachinelearning/webnn/issues/181 Issue #181 14:22:41 -> https://github.com/webmachinelearning/webnn/pull/195 PR #195 14:22:59 anssik: HardSwish is an important activation function in MobileNetV3. Both TensorFlow and ONNX have supported it. 14:23:22 ... In the issue we had discussion on whether this op should be a composite of existing ops. A few alternatives for compositing discussed in this issue. 14:23:39 ... the research done in the issue suggests a number of native ML APIs support hard-swish, and the op is used by first-wave models. 14:24:02 ... We've usually used these criteria in evaluating whether to include a new op into the spec 14:24:16 ... Given this op passes this test, a PR was submitted, reviewed and merged to add HardSwish op to the API spec in PR #195 14:24:28 anssik: Ningxin, you want to talk about this op? 14:24:41 ningxin_hu: thanks Anssi, good summary 14:25:11 ... thanks Chai for review and suggestion to be careful about the term and use a more generic terminology in this op speccing 14:25:52 ... we have WebNN polyfill with tests, I'd like to say Chai's PR for fused activations is already landed in the WebNN polyfill 14:26:31 ... this polyfill is based on TF.js and we utilize TF fused ops within the polyfill and get good speedup 5-7x in MobileNetV2 with these fused ops and Wasm backend 14:26:48 ... a good example of the performance benefit of adding this type of op 14:27:05 q+ 14:27:21 ack Chai 14:28:35 Chai: it is useful to state the criteria of defining new ops, if you look at the discussion between myself and Ningxin in the issue, we are saying if there's a new proposal for adding a new op, we want to look if the native platforms support it, then we should define the op, because the goal is to accelerate using the native ops 14:28:58 ... on the flipside of the argument is, what if a certain platform cannot accelerate the op? 14:29:10 ... the general answer is we emulate 14:29:42 ... in those cases for implementation that don't do acceleration, the implementation should then decompose the ops into subgraphs, that is what we were discussing in the issue 14:30:28 anssik: NOTE TO SELF: Discuss/document the rationale/criteria for adding new ops to the spec makes for a great TPAC WG meeting discussion topic 14:31:31 Chai: e.g. GRU really is a graph that can always emulated, the questions is whether it can be accelerated, we need to look for evidence an op can be accelerated, if none found we should not define it in the spec 14:32:18 +1 to document the criteria for new ops 14:32:37 New op: 1. can it be emulated?, 2) can it be accelerated? 14:33:07 ... Those two PRs were the new features that landed since our last meeting. We'll discuss WebNN API recent new feature requests next. 14:33:11 RRSAgent, draft minutes 14:33:11 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 14:33:18 Topic: WebNN API recent new feature requests 14:33:28 Subtopic: BatchNormalization should be an optional operation 14:33:36 -> https://github.com/webmachinelearning/webnn/issues/187 Issue #187 14:33:42 anssik: Ping opened an issue, quoting: 14:33:47 ... "BatchNormalization usually are optimized away for inference (fused into conv2d op filter and bias), for example TFLite does not have this op, for compatibility purpose we shall move this op to optional section of the spec." 14:34:40 Ping: for PatchNorm, many of our models are optimized and fused, especially for inference purposes, e.g. TF.js does not have this supported 14:34:56 ... question is whether this op should be optional 14:35:37 ... there's also some discussion, Jonathan is not here today, here's on vacation, so maybe we want to postpone this for next week 14:35:59 anssik: Chai suggests BatchNorm is supported as a native operation in DirectML 14:36:20 ... noting a backend implementation should be allowed to treat BatchNorm as a standalone operation, or fuse it with just the following relu, etc. 14:37:00 Chai: my feedback is in the issue, related to the logic discussed in previous topic, it can be emulated and accelerated, so think we should define it 14:37:19 ... if a platform cannot support it, it can emulate it 14:37:47 ... technically speaking WebNN can be defined without any big ops, but we want to define them due to acceleration, the point we want WebNN to be fast 14:38:02 ... it is supported in all recent GPU drivers 14:38:31 ... what Ping brought up, optionality is related but different, discussion 14:39:18 ... if someone is to implement WebNN, the users don't know ahead of time what ops are optional and what required, so hard to rely on these ops, from user's point of view optionality makes it hard to code against the API 14:39:26 q? 14:40:04 q? 14:43:39 Ping: thanks Chai, thinking further, optionality is bad for users, what i the standard best practice for future additions? versioning? 14:43:47 s/i the/is the 14:44:41 q+ 14:44:46 ack Chai 14:45:05 Chai: thanks Ping, fusion is probably one of the big topics in MLGraph 14:45:32 ... it is complicated and open-ended, your question is what is the philosophy around the API, and versioning is a great topic for discussion 14:46:11 ... on the topic of fusion, it is open-ended, anyone can argue that some implementation can fuse the entire graph into one thing if they control the whole SW/HW stack 14:46:40 ... fuse op in activation, I think we want to make it scoped, every implementation already takes care when you can conv followed by a relu 14:47:08 ... so in a way it is already a special case, similarly batchnorm followed by relu, the platform will in reality fuse them 14:47:29 ... the change to support fused ops is a reaction to that trend, but a very scoped fusion 14:47:53 ... a more generic fusion, graph with conv, batchnorm and relu, you want these fused 14:48:16 ... do we want to go so far we want to make batchnorm a fused op to be passed to conv2d 14:48:31 ... it may get very complicated, with circular deps 14:49:03 ... how to deal with this, we already have a provision for complete step, for open-ended hw specific ones compile step takes care of this 14:49:51 ... the fused activation should be used in very limited way when we know if can be supported in modern platforms, otherwise use complication step 14:51:09 anssik: NOTE TO SELF: TPAC topic proposal: "How to WebNN deal with fusion" 14:51:50 Subtopic: BatchToSpaceND and SpaceToBatchND ops 14:51:54 -> https://github.com/webmachinelearning/webnn/issues/189 Issue #189 14:52:21 anssik: this request is motivated on: "SRGAN model contains BatchToSpaceNd op, can we support it in WebNN? And SpaceToBatchND as well. In ONNX, similar ops are DepthToSpace and SpaceToDepth." 14:52:26 ... No reactions yet on GH. Please provide your feedback on the issue and we'll bring this to our upcoming calls. Any quick comments? 14:52:38 q? 14:53:04 Subtopic: The scales and sizes of MLResampleOptions can't be both empty 14:53:16 -> https://github.com/webmachinelearning/webnn/issues/192 Issue #192 14:53:35 anssik: quoting the issue author: "scales and sizes are hosted in the MLResampleOptions, both of them could be empty. I think when neither scales nor sizes is defined, the resample operation shouldn't be created." 14:53:44 it looks like a spec bug 14:53:46 ... I believe we need to spec what is returned by resample() under such conditions. 14:54:14 q? 14:54:29 Subtopic: Support for configuring rounding type in pooling operations 14:54:36 -> https://github.com/webmachinelearning/webnn/issues/198 Issue #198 14:54:59 anssik: While implementing OpenCV pooling layer with WebNN backend, it was observed OpenCV's pooling layer supports both ceil (with ceilMode flag) and floor rounding type 14:55:16 ... the issue is, WebNN API spec does not support configuring the rounding type in pooling operations, so the author suggests considering adding support for configuring rounding type 14:55:26 ... also preliminary investigation suggests ONNX pooling also supports both ceilMode and floorMode 14:55:32 ... thoughts? 14:56:07 q+ 14:56:11 ack ningxin_hu 14:56:59 ningxin_hu: I'm the mentor of the GSoC project with the student, working on the OpenCV DL module 14:57:36 ... proposed next step to investigate this request based on criteria we've developed to see if it can be emulated, and whether native APIs can accelerate 14:58:18 ... I'm committed to work with the student on this issue 15:00:32 anssik: Thank you all for joining! 15:00:53 ... we'll defer WebNN API TAG review issue discussion to our next call 15:00:58 RRSAgent, draft minutes 15:00:58 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 15:08:52 s/How to WebNN/How should WebNN 15:08:54 RRSAgent, draft minutes 15:08:54 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 15:09:26 s/NOTE TO SELF/TPAC topic proposal 15:09:32 RRSAgent, draft minutes 15:09:32 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 15:10:40 s/TPAC topic proposal: "/" 15:10:43 RRSAgent, draft minutes 15:10:43 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 15:11:34 s/NOTE TO SELF/TPAC topic proposal 15:11:35 RRSAgent, draft minutes 15:11:35 I have made the request to generate https://www.w3.org/2021/08/19-webmachinelearning-minutes.html anssik 17:12:30 Zakim has left #webmachinelearning