13:53:46 RRSAgent has joined #webmachinelearning 13:53:46 logging to https://www.w3.org/2021/06/24-webmachinelearning-irc 13:53:48 RRSAgent, make logs Public 13:53:50 please title this meeting ("meeting: ..."), anssik 13:53:55 Meeting: WebML WG Teleconference – 24 June 2021 13:54:01 Chair: Anssi 13:54:13 Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2021-06-24-agenda.md 13:54:42 Scribe: Anssi 13:54:44 scribeNick: anssik 13:54:56 Present+ Anssi_Kostiainen 13:55:26 scribe+ 13:55:45 RRSAgent, draft minutes 13:55:45 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html anssik 13:58:15 Present+ Sungpil_Shin 14:00:16 ningxin_hu has joined #webmachinelearning 14:00:31 Sungpil_Shin has joined #webmachinelearning 14:00:36 Present+ Dominique_Hazael-Massieux 14:00:45 Present+ Chai_Chaoweeraprasit 14:00:50 Present+ Ningxin_Hu 14:00:51 chai has joined #webmachinelearning 14:01:43 Rama has joined #webmachinelearning 14:02:01 Present+ Wonsuk_Lee 14:02:11 Jonathan has joined #webmachinelearning 14:02:12 Present+ Rafael_Cintron 14:02:19 Present+ Jonathan_Bingham 14:02:30 Present+ Ganesan_Ramalingam 14:02:39 RRSAgent, draft minutes 14:02:39 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html anssik 14:03:05 wonsuk_ has joined #webmachinelearning 14:03:13 Topic: information_source WebNN API - First Public Working Draft 14:03:25 s/ information_source// 14:03:46 Anssi: welcome to the 2nd call of the Web Machine Learning WG 14:04:04 ... a few new names - intros? 14:04:36 SungpilShin_ETRI: I'm joining to take part to the work on Machine Learning 14:04:40 ... I work with Wonsuk 14:05:09 ... we're interested about unew se cases for the Web Machine Learning 14:05:14 ... that I would like to see discussed 14:05:22 -> https://github.com/webmachinelearning/webnn/pull/182 Add a new application use case #182 14:05:35 Sungpil: we plan to contribute to the APIs of this WG 14:05:46 Anssi: welcome to the group! 14:06:01 s/WebNN API - First Public Working Draft/Intros/ 14:06:10 Topic: WebNN API - First Public Working Draft 14:06:20 Anssi: WebNN was published as First Public Working Draft on June 22nd 14:06:25 -> https://www.w3.org/TR/2021/WD-webnn-20210622/ Web Neural Network API 14:06:25 W3C First Public Working Draft, 22 June 2021 14:06:37 ... congrats to the group on this milestone! 14:06:43 anssik: "The first Working Draft of a technical report is called the First Public Working Draft (FPWD), and has patent implications as defined in the W3C Patent Policy [PATENT-POLICY]." 14:07:04 RafaelCintron has joined #webmachinelearning 14:07:50 ... our first step towards Recommendation 14:07:56 -> https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2021Jun/0013.html Call for Exclusions: Web Neural Network API 14:08:08 ... in terms of the RF licensing, you received a call for exclusions email 14:08:22 ... all Advisory Committee representatives were copied - they are the ones in charge of that process 14:09:33 Anssi: Google provided feedback in the Call for Consensus to move forward with FPWD publication 14:09:58 -> https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2021Jun/0009.html Google's input on WebNN CfC to FPWD 14:10:13 Anssi: recommendation to pay attention to rapid domain evolutions 14:10:19 ... pay attention to developer demands 14:10:40 ... take into account lessons from real-world benchmarks 14:10:51 ... and pay attention to advances in complementary areas such as Web Assembly 14:11:27 ... thanks a lot for that feedback, Jonathan - great input to building a consensus view of our work ahead 14:11:53 Jonathan: good summary - these aren't new concerns, we've mentioned these points before 14:12:46 ... we recognize that there is value in providing a graph-based API despite other advances we've seen, given the time it takes to deploy a Web standard and given the expected benefits of an improved primitive 14:13:34 ... multiple people have skepticism this is the long-term optimal approach, but we have good hopes that it can be at least a good stepping stone, if not the ultimate solution 14:13:46 +1 14:13:50 +q 14:13:52 q+ 14:14:15 ack RafaelCintron 14:14:17 ack RafaelCintron 14:14:47 RafaelCintron: will Google invest in WebNN given the possible mitigations you mentioned? 14:15:05 Jonathan: one of the risks we had identified is the growth of operation set 14:15:17 ... due to operations not being composable based on existing operations 14:15:25 Sungpil_Shin_ has joined #webmachinelearning 14:15:32 ... tensorflow is not seeing any slow-down in that growth at that moment 14:15:53 ... the mitigation we're exploring in Google is finding the right set of lower level operations that can be used to compose other operations 14:16:02 ... we're hopeful this can be achieved 14:16:16 ... the NN API group is still working on the next iteration of that API 14:16:26 ... a complete revamp of the existing API 14:17:00 ... I'm not sure why they didn't complete the existing NN API with lower level APIs - will share more when I know more 14:17:11 Anssi: NN API is the Android NN API here 14:17:25 RafaelCintron: will it be a graph API or is still TBD? 14:17:42 s/WebNN/in NN API as a graph API/ 14:18:01 q+ for plan for implementation on Chromium 14:18:08 ack wonsuk_ 14:18:08 wonsuk_, you wanted to discuss plan for implementation on Chromium 14:18:37 wonsuk_: are there plans to implement WebNN in Chromium? 14:19:15 Jonathan: right now the focus in Chrome is on the Model Loader API - separate from the work on NN in Android 14:19:34 ... eventually, there may be different implementations across PC vs android 14:19:45 Anssi: implementations is also part of the next agenda item 14:19:51 q? 14:20:11 -> https://www.w3.org/2020/Process-20200915/#w3c-recommendation-track W3C Recommendation Track flow diagram 14:20:13 Subtopic: Next steps 14:20:39 Anssi: the first step is what we've just done, FPWD 14:20:47 ping_yu has joined #webmachinelearning 14:20:54 ... next step is publication of 0 or more revised WDs 14:21:03 -> https://lists.w3.org/Archives/Public/public-webmachinelearning-wg/2021Jun/0011.html Automating publication of future WebNN Working Drafts 14:21:15 ... we're set up with an automated publication system which allows us to publish WDs on a regular basis 14:21:23 ... each merged Pull request will trigger a new WD publication 14:21:49 ... WD are development versions of the spec, they don't necessarily represent consensus 14:22:07 ... the following step is Candidate Recommendation (CR) 14:22:28 ... reaching that stage requires fulfilling the technical requirements 14:22:58 ... it's a signal to get a final review of the spec and a signal the spec should be implemented 14:23:13 ... nowadays, in practice, implementations often start well ahead of CR 14:23:25 ... and we expect to get wide review from the community before we reach CR 14:23:31 ... CR is the time when this gets assessed 14:24:35 dom: horizontal group review expected latest at CR 14:25:17 ... another aspect is testing the API, part of what is assessed during CR is how much implementation we're getting, web-platform-tests is the project that hosts cross-browser tests 14:25:51 ... tests help implementers implement and assess implementation gaps 14:26:47 Anssi: the WebNN polyfill has a test suite - that might help down the line. Ningxin, where does that stand? 14:27:04 Ningxin: the webnn polyfill has a tests folder with mocha tests 14:27:15 -> https://github.com/webmachinelearning/webnn-polyfill/tree/master/test webnn-polyfill mocha tests 14:27:30 ... we have an open issue to turn these tests into WPT tests 14:27:55 -> https://webmachinelearning.github.io/webnn-polyfill/test/ Test runner 14:28:08 ... the porting of the test framework is still TBD, but the test cases themselves should be in a good shape 14:28:28 ... the tests exercise at @@@ and operators 14:28:53 ... there was discussion in the workshop around testing, with chai giving very good advice 14:29:00 ... we have tests both at the operator and model levels 14:29:31 -> https://github.com/w3c/machine-learning-workshop/issues/80 Conformance testing of ML APIs for the Web (from W3C workshop) 14:29:59 ... we have ported some of the native tests e.g. from Android NN 14:30:03 ... and ONNX 14:30:20 ... includes different memory layout to increase our coverage 14:30:27 ... it's an ongoing effort 14:30:41 RRSAgent, draft minutes 14:30:41 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html anssik 14:30:47 ... we have a dozen tests for ops - the coverage is far from being complete 14:30:55 ... contributions are highly welcomed! 14:31:58 Anssi: are we expected to use the WPT test harness? 14:32:47 dom: in practice it really helps to get browsers run these tests in their CI 14:33:32 Anssi: so translating the tests sounds like an important thing to do - is that something you can take on Ningxin? 14:33:37 Ningxin: will do 14:34:20 Anssi: in our charter, our timeline for reaching CR would be Q2 2022 - naturally, this is subject to our progress, based on consensus 14:34:44 ... After CR, next step is Proposed Recommendation, the penultimate stage before the final status of Recommendation 14:34:59 ... it triggers formal review by the W3C Advisory Committee, the whole W3C Membership 14:35:46 q? 14:35:54 Dom: the real challenges are in getting into CR and out of CR - the rest is usually more of a formality 14:36:03 Topic: Supporting implementation efforts: webnn-native and webnn-polyfill 14:36:44 Anssi: Ningxin, can you talk about webnn-native and -polyfill, software projects from our sister Community Group? 14:36:52 ... they inform the work on the WebNN API 14:37:08 ... I expect we'll discuss the progress of these projects on these calls from time to time 14:37:32 ... useful to highlight where contributions are welcomed in particular 14:37:43 -> https://github.com/webmachinelearning/webnn-polyfill webnn-polyfill 14:37:47 Subtopic: WebNN Polyfill 14:38:28 Ningxin: the polyfill was started first - it gives a JS implementation of the API, based on tensorflow.js - thanks to Ping and his team for their help 14:38:54 ... polyfills help with early implementation and usage experience 14:39:15 ... as we discussed, the test suite was built in the context of the development of this polyfill 14:39:25 -> https://webmachinelearning.github.io/webnn-samples-intro/ Samples that run with webnn-polyfill 14:39:39 ... In terms of contribution areas, the major thing is to keep alignment with API changes 14:39:46 -> https://webmachinelearning.github.io/webnn-polyfill/test/ Test runner for webnn-polyfill 14:39:59 ... e.g. recent device selection, pre-allocated output buffers, new operations 14:40:18 ... we want to keep the polyfill as close as possible to the spec definition 14:40:52 -> https://github.com/webmachinelearning/webnn-polyfill/issues webnn-polyfill open issues 14:40:56 ... this is reflected in the open issues of the repo 14:41:43 https://github.com/webmachinelearning/webnn-samples webnn-samples GH repo 14:41:44 ... we also have sample codes in the spec & in the explainer - they can be run through the polyfill which are made available through the webnn-samples-intro repository 14:41:52 s/samples-intro/samples/ 14:42:14 -> https://webmachinelearning.github.io/webnn-samples/code/ WebNN Code Editor 14:42:29 ... good way to explore new use cases for the API 14:43:13 Anssi: the polyfill is a good way to explore the API, easy to integrate into one's application 14:43:15 ... very useful project 14:43:26 ... many of the samples perform quite nicely with the polyfill 14:43:42 q? 14:43:51 Subtopic: webnn-native 14:43:54 -> https://github.com/webmachinelearning/webnn-native webnn-native 14:43:57 RRSAgent, draft minutes 14:43:57 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html dom 14:44:16 Ningxin: we started webnn-native with support from the Community Group 14:44:37 ... the initial intention is to inform the CG (and now the WG as well) for the WebNN API design 14:44:45 ... also helps with performance investigation 14:45:29 ... it also provides C & C++ headers for applications to use, allowing WebAssembly use 14:45:51 ... can be useful in transpiled projects like TensorFlow lite & ONNX 14:46:02 -> https://github.com/webmachinelearning/webnn-native#readme webnn-native readme 14:46:15 ... the repo has different backend implementations - DirectML, OpenVino 14:47:01 ... it allowed running benchmarks with recent changes to validate them 14:47:31 -> https://github.com/webmachinelearning/webnn-native/issues webnn-native open issues 14:47:35 ... In terms of opportunities for contributions, looking at open issues, it's again keeping the implementation close to the spec 14:47:54 ... right now, it's a bit behind than the polyfill - implementation is a bit more involved than a JS implementation 14:48:18 -> https://github.com/webmachinelearning/webnn-native/pulls Node.js binding to webnn-native 14:48:47 ... enabling running the tests would help determine conformance across different backends 14:49:05 ... there are also backend-specific tasks - thanks to Chai and his team for their help on the DirectML backend 14:49:46 ... one possible improvement would be to remove duplication with DirectML by upstreaming some of the code there 14:50:26 Backend implementations that use platforms' ML APIs: 14:50:26 DirectML on Windows 10 14:50:26 OpenVINO on Windows 10 and Linux 14:50:26 oneDNN on Windows 10 and Linux 14:50:26 XNNPACK on Windows 10 and Linux 14:50:26 Other backends are to be added 14:50:47 ... additional backends for other OS native APIs would be highly welcome 14:51:10 q? 14:51:15 ... combined with the tests, this would help detect good cross-platform support for operators 14:51:44 Anssi: both projects keep track of the spec changes closely, as reflected in issues 14:51:52 q? 14:52:34 ... webnn-native could be used as a backend to a chromium implementation 14:52:51 Wonsuk: amazing work, with both projects! 14:53:09 ... thanks for your efforts! 14:53:30 q? 14:53:55 ... I'll try to get started with running the tests 14:53:58 q? 14:54:09 Topic: WebNN API - TAG review 14:54:30 Anssi: WebNN has received review from the W3C Technical Architecture Group 14:54:41 Subtopic: [tag-tracker] Define a common term for logical tensor changes? 14:54:42 ... most of the issues have been addressed, a few are still being worked on 14:54:46 -> https://github.com/webmachinelearning/webnn/issues/150 Define a common term for logical tensor changes? #150 14:55:21 Anssi: Rama, you had a proposal to address this - would be good to turn it into a pull request, a good first contribution 14:55:31 Subtopic: [tag-tracker] Ergonomics of the JS examples 14:55:38 -> https://github.com/webmachinelearning/webnn/issues/139 Ergonomics of the JS examples #139 14:55:59 ... "RESOLUTION: Explain in the spec intro the rationale why the primary API consumer is a JS framework, note Model Loader API as a higher-level abtraction targeting web developers" 14:56:04 -> https://www.w3.org/2021/04/15-webmachinelearning-minutes.html#r03 RESOLUTION from 2021-04-15 14:56:43 sgtm 14:56:43 Anssi: the main story here is that the main users of the spec are expected to be framework developers, with a higher level API exposed to regular developers, e.g. model loader 14:56:48 Subtopic: [tag-tracker] String enum for activations 14:56:55 -> https://github.com/webmachinelearning/webnn/issues/138 String enum for activations #138 14:57:10 anssik: no new information, Chai feel free to submit a PR 14:57:40 chai: working on this as part of the discussion on activation functions 14:58:06 anssik: great! finalizing our responses to these issues would be good 14:58:13 Topic: Help wanted! 14:58:31 Anssi: there are issues in our tracker that are welcoming comments and proposed solutions 14:59:03 ... please review the list for triage purpose - I did some initial triage, classifying them into 5 buckets: proposed new features, to be clarified, ready to close, investigation and explainer updates 14:59:10 -> https://github.com/webmachinelearning/meetings/blob/main/telcons/2021-06-24-agenda.md#%E2%84%B9%EF%B8%8F-help-wanted Help wanted issues 14:59:24 ... please take a look and provide feedback 14:59:43 ... if you have a solution in mind, please provide a pull request - the editors can help iterate 14:59:51 q+ 15:00:34 im working on a PR for #138, 181, 184 and 185 15:00:35 ... there are a bunch of new ops being proposed for which we would need to look at decomposition, platform support, … 15:00:52 ack wonsuk_ 15:01:37 wonsuk: sungpil made a PR for a new use case for WebNN - given the time, we won't be able to discuss it today, so please look at the PR and give your opinions 15:01:56 https://github.com/webmachinelearning/webnn/pull/182 15:02:09 Anssi: I'll take a look 15:02:16 RRSAgent, draft minutes 15:02:16 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html dom 15:02:53 Topic: Summer break 15:03:00 anssik: These meetings will break for July and will resume mid-August 15:03:08 ... Please respond to the meeting scheduling poll to signal your preferences: 15:03:16 -> https://doodle.com/poll/gymnpmtcu4dcknck WebML WG Teleconference scheduling poll 15:03:40 anssik: Note: the request is for a recurrent meetings, not the specific week (frequency to be confirmed, likely every other week for the time being) 15:04:23 ... The GitHub repos will remain open for contributions while our meetings are on pause. Thank you for your continued contributions! 15:05:40 ... Those of you who take time off: have a happy vacation, stay safe and enjoy! 15:05:56 Thanks Anssi. Enjoy the summer everyone! 15:06:09 RRSAgent, draft minutes 15:06:09 I have made the request to generate https://www.w3.org/2021/06/24-webmachinelearning-minutes.html anssik 16:59:58 zkis has joined #webmachinelearning 17:26:41 Zakim has left #webmachinelearning