13:59:03 RRSAgent has joined #webmachinelearning 13:59:03 logging to https://www.w3.org/2021/04/29-webmachinelearning-irc 13:59:06 RRSAgent, make logs Public 13:59:07 please title this meeting ("meeting: ..."), anssik 13:59:12 Meeting: WebML CG Teleconference – 29 April 2021 13:59:19 Chair: Anssi 13:59:20 dom has joined #webmachinelearning 13:59:26 Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2021-04-29-agenda.md 13:59:34 jonathan has joined #webmachinelearning 13:59:59 Scribe: Anssi 14:00:04 scribeNick: anssik 14:00:11 Present+ Anssi_Kostiainen 14:00:17 Present+ Dominique_Hazael-Massieux 14:00:21 Present+ Jonathan_Bingham 14:00:52 ping_yu has joined #webmachinelearning 14:00:58 Present+ Rafael_Cintron 14:01:06 Present+ Ping_Yu 14:01:26 ningxin_hu has joined #webmachinelearning 14:01:54 rama has joined #webmachinelearning 14:02:14 scribe+ 14:02:26 Present+ Ganesan_Ramalingam 14:02:50 RRSAgent, draft minutes 14:02:50 I have made the request to generate https://www.w3.org/2021/04/29-webmachinelearning-minutes.html anssik 14:02:50 Geun-Hyung has joined #webmachinelearning 14:03:13 Present+ Geun-Hyung 14:03:33 Topic: Web Machine Learning WG launched 14:04:00 anssik: thanks to your contributions over the two+ years including supportive review from the W3C Advisory Committee, W3C launched the Web Machine Learning Working Group last week -- congrats y'all! 14:04:05 -> https://lists.w3.org/Archives/Public/public-webmachinelearning/2021Apr/0005.html Call for Participation: Web Machine Learning Working Group 14:04:16 -> https://www.w3.org/blog/2021/04/w3c-launches-the-web-machine-learning-working-group/ Annoucement blog post 14:04:35 ... your actions: click the "Instructions for joining the Web Machine Learning Working Group" link and follow the instructions to join the WG: 14:04:42 -> https://www.w3.org/2004/01/pp-impl/130674/instructions Instructions for joining the Web Machine Learning Working Group 14:05:01 ... some of you have already joined. 14:05:05 RafaelCintron has joined #webmachinelearning 14:05:16 ... your AC rep will get notified of your request and will initiate the internal reviews and will join you 14:05:31 ... internal reviews might take some time and thus we wait for the critical mass of key contributors to join before we hand over the WebNN API to the WG, meanwhile we're advance the API in this CG 14:06:05 Dom: thanks to the Community Group for getting to the Working Group! Nice job! 14:06:58 ... I put a few proposed topics to discuss to the agenda: 14:07:08 anssik: 1) Discuss how this Community Group and the new Working Group will work together 14:07:23 ... 2) Discuss how we plan to take the Web Neural Network API to First Public Working Draft and beyond 14:07:34 ... 3) Discuss the new new deliverable working title "Ethical Impact of Machine Learning on the Web" 14:08:15 Anssik: WG/CG collaboration - short answer is Dom will take care of the transition & collaboration aspects so y'all can focus on the technical work 14:08:21 anssik: starting with (1), the short answer is with Dom we'll take care of the transition aspects so you can focus on the technical work. In short, the transition should be almost transparent to you. The spec will adopt the official W3C spec format which changes some visual aspects. 14:08:23 ... the spec will evolve to take the WG style 14:08:59 Dom: as we transition to a WG, there's more attention to the work from broader community 14:09:13 ... we may need to adjust some logistics e.g. meeting time rescheduling 14:09:19 Present+ Zoltan_Kis 14:10:05 anssik: as for (2), we're a path to publish the WebNN API First Public Working Draft (aka FPWD) during Q2 2021 14:10:12 chai has joined #webmachinelearning 14:10:27 ... given this is the first formal publication, no "previous maturity level" exists and many requirements do not apply. Later stages add more requirements, esp when transitioning to Candidate or Proposed Recommendation. 14:10:48 ... given we've been proactive and initiated both TAG and PING reviews, addressing those issues by the time we publish FPWD would be great demonstration of early wide review 14:10:56 ... questions about the WebNN API First Public Working Draft? 14:11:12 Present+ Chai_Chaoweeraprasit 14:11:45 Dom: before WG takes over CG spec, CG should publish a formal CG report and make final licensing request? 14:12:13 ... questions about the WebNN API First Public Working Draft? 14:12:30 anssik: about (3), per AC review comments, the WG added a new deliverables working title "Ethical Impact of Machine Learning on the Web" 14:12:36 ... the charter says: "The Working Group will develop a Working Group Note documenting ethical issues associated with using Machine Learning on the Web, to help identify what mitigations its normative specifications should take into account." 14:13:09 anssik: I propose we create a repo for this deliverable. I'm volunteering to bootstrap the work on this deliverable and welcome your contributions. I was planning to structure the doc around the following high-level topics each documenting the potential concerns: 14:13:16 ... 1) Bias - how AI systems may be unfair, or have harmful impacts to certain groups of people due to social differences 14:13:42 ... 2) Privacy - obviously example is face recognition 14:14:37 ... 3) Explainability and Transparency - how data is gathered, how algos are trained and tuned, web tech can be part of the solution helping visualize models, but DL network as not very explainable due to complexities. 14:14:50 ... repo name proposal: "ethical-webmachinelearning"? 14:15:58 Dom: thanks for the early proposal, structurally useful to identify if there are well-known taxonomy ethical risks we could build upon 14:16:12 ... or communities we could ask for input as we work on this document 14:16:27 ... similarly to accessibility, internationalization etc. 14:16:40 ... as a way to anchor existing work into this framework 14:16:42 +1 anssik 14:17:44 q? 14:18:01 Topic: Operation-specific APIs proposal 14:18:05 zkis has joined #webmachinelearning 14:18:14 -> https://github.com/webmachinelearning/proposals/issues/2 Operation-specific APIs proposal 14:18:25 anssik: Continue refine work-in-progress features, A LOT of good discussions in PR #162 14:19:07 Subtopic: WebAssembly op level execution use case and resource upload and download 14:19:13 -> https://github.com/webmachinelearning/webnn/issues/156 Issue #156 14:19:17 -> https://github.com/webmachinelearning/webnn/pull/162 PR #162 14:19:29 anssik: PR #162 is a substantial one with 50+ comments and deep design discussions going on in it. Thanks Chai, Ningxin, Ping, Rama, others for work on this. 14:20:36 Chai: there's too many topics in this PR, should maybe be split 14:20:57 ... two parts: device selection and data download and upload and layout conversion 14:21:08 ... first part almost consensus, except naming 14:21:20 -> https://github.com/webmachinelearning/webnn/pull/162#discussion_r612914114 Device preference naming 14:21:38 ... - device preference naming suggestions "software", "hardware", or "cpu", "gpu", "accelerator" / "custom" etc. 14:22:22 Chai: two sub-issues: naming and whether or not we want to name the accel 14:22:44 ... in my PR the enum is "default", "cpu", "gpu", "accelerator" 14:22:58 accelerator is the hardest, CPU and GPU are well-defined 14:23:17 ... if the define is a CPU device you know use use ArrayBuffer, for GPU maps to WebGL and GPU 14:23:19 q+ 14:23:55 ... accelerometer has not been defined, e.g. DirectX may decide to map its API to accelerometer, then you could use WebGL contract to use that accelerator undernearth, say Intel VPU 14:24:10 ... WebGL term is a bit weird in this context 14:24:19 q? 14:25:05 ... we already have an enum to control the power profile, one can still create a selection policy that maps to accelerometer 14:25:07 q? 14:25:54 q? 14:25:57 ack zkis 14:26:15 Zoltan: wanted to note Kubernetes struggles with the same problem 14:26:23 ... used labels and selectors instead 14:26:52 ... vendor independent way to map to particular device types 14:26:55 from usability point of view, it would be nice for the developer to query the availability of devices 14:27:30 [querying available devices tends to get serious frowns from a privacy perspective] 14:27:53 https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/ 14:27:58 -> https://w3ctag.github.io/design-principles/#naming-is-hard Web Platform Design Principles > Naming principles 14:28:31 anssik: could we spin naming into its own issue? 14:28:56 Chai: we could to that, remove accel from the PR for now 14:29:30 need to step away a bit 14:29:56 Ningxin: agree with Chai that naming is still open, would like to say for this issue, device selection, major open issue to close is to make sure resource UL/DL is effective 14:30:02 q+ 14:30:10 ... if we select GPU then developers should know they're using GPU Resource 14:30:27 ack RafaelCintron 14:30:52 RafaelCintron: WebGPU CG has been tackling this at the very last meeting 14:31:05 ... they have requestAdaptor that accepts filters one is powerPreference 14:31:26 ... recently struggling if I'm building an app I know will run badly on software, how to filter that out 14:31:46 ... adapter can be plular, can be multiple and there's an attribute that tells whether an adapter is software 14:32:09 ... generally supportive of this, Apple has some concerns about this design in WebGPU discussion quoting privacy reasons 14:32:48 ... enumerate and filter devices needed for compute-only devices in WebGPU in particular 14:32:52 q? 14:32:56 q+ 14:32:59 q? 14:33:05 q? 14:33:07 ack dom 14:33:37 Dom: privacy challenge noted by Rafael is an important and recurrent theme with device selection and enumeration 14:33:42 WebGPU PR#1: Pluralize requestAdapters() and add GPUAdapter.isSoftware (https://github.com/gpuweb/gpuweb/pull/1660) 14:34:12 WebGPU Issue: Add GPURequestAdapterOptions.allowSoftware (https://github.com/gpuweb/gpuweb/pull/1634) 14:34:24 ... privacy and fingerprinting also impacting WebRTC API that had to refactor its enumeration API 14:34:58 ... a word of caution, it is not just about dev experience, so any decision in this space need to be careful and balance with DX 14:34:59 q? 14:35:08 q+ 14:35:13 ack zkis 14:35:38 Zoltan: I agree we should have a separate discussion, when we speak about privacy should keep threat mitigation discussion concrete 14:35:58 ... having a "CPU" is not a privacy issue 14:36:06 +1 to zoltan 14:37:04 -> https://github.com/webmachinelearning/webnn/pull/162#discussion_r613439361 behavior of MLTensor for the pre-allocated outputs scenario 14:37:37 q+ 14:37:41 ack ningxin_hu 14:38:04 ningxin_hu: I think this discussion moves to a later one with Chai, Rama, Anssi 14:38:08 https://github.com/webmachinelearning/webnn/pull/162#discussion_r613729314 14:39:39 ningxin_hu: major open for this comment, before this PR we had resource binding for inputs and outputs 14:40:10 ... compute() would layout convert output data to ArrayBuffer, with this PR Chai introduced MLTensor with sync data() method to help developers 14:40:29 ... this created some open with resource binding mechanism 14:40:57 ... i.e. if you have ArrayBuffer binding and another API with async binding, it seems complex 14:41:10 ... made a proposal to separate MLTensor from GPU Resource binding 14:41:28 q+ 14:41:29 ... MLTensor seems like an upload source and download target 14:41:48 Another link on kubernetes (to the previous topic): https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/ 14:42:06 ... MLTensor does not then have any binding to ArrayBuffer as before, it'd satisfy different device type requirements from the table in this PR 14:42:07 q? 14:42:18 ... would make the spec self-contained 14:42:49 ... also proposed GPU Resource is a binding mechanism, I tend to propose to use the previous mechanism, like we do for MLInput and MLOutput binding, would work well for GPU Resources 14:43:23 ... also mentioned GPU Resources have their own WebGL and GPU APIs for UL/DL, looks like we can use those APIs for UL/DL instead of introducing a new similar mechanism in WebNN API 14:43:24 q? 14:43:25 q+ 14:43:27 ack RafaelCintron 14:43:49 RafaelCintron: question about MLTensor proposal: what point is the ArrayBuffer copied into MLTensor? 14:43:55 (need to step out for a few mins) 14:43:58 s/what point/at what point 14:44:04 q? 14:44:37 (back now) 14:45:09 ningxin_hu: not specified yet(?), discussion focused on the download part now 14:45:12 q? 14:45:23 ack chai 14:45:25 q? 14:45:52 Chai: seems to be the same issue, Ningxin's explanation was a great example of complexities of data UL/DL with different resource types 14:46:10 ... my opinion is much of the discussion in this PR is around this complexity 14:46:32 ... should be able to manage data download outside WebNN API leave that to the framework 14:46:48 ... the more we try to abstract this, the more edge cases you'll find 14:47:00 ... the caller is not the one we assume, major discussion point in this PR 14:47:21 ... the main goals of the WebNN should be I'm binding this resource to the WebNN API 14:47:46 ... once the output resource is bound to the graph and it is executed, it is up to the caller 14:48:05 ... the caller known with which type of resource it deals with, can do overlapping upload and download 14:48:35 ... you may want to in parallel upload the next frame and let caller manage overlapping DL/UL e.g. on different threads 14:48:45 ... there's still confusion around it, I'm sure 14:49:06 ... I hear Rafael asking about data upload, it is the same problem, sync of resources and timing 14:49:15 ... the more we abstract, the more challenging 14:49:25 Rafael: leave it to the user? how does that change the API? 14:49:34 Chai: we may not need MLTensor as in the PR now? 14:49:55 ... also async call can be eliminated, we go back to the earlier design before this PR 14:50:15 Rafael: how about weights? 14:50:27 Chai: weights are operands 14:51:05 Rafael: values of the weights? 14:51:37 Chai: whatever is given to the model, DirectML that runs in the backend needs to negotiate with the driver, 10/10 cases the driver will need to convert the format into native layout 14:51:42 ... cannot happen in the userspace 14:52:04 ... bind the weights with the model, when compiling the graph, all the weights are compiled to a native format 14:52:22 RafaelCintron: what is the data type for weights web devs give? 14:52:35 Chai: initially ArrayBuffer mapped to CPU 14:53:02 ... caller can interact with WebGPU, if you get model out and upload everything to GPU, then the weights are already in the GPU memory 14:53:37 ... WebNN should just offers means to bind the resources 14:53:39 q? 14:53:43 q+ 14:53:47 ack ningxin_hu 14:54:06 ningxin_hu: I'm hearing Chai suggests to go back to the previous MLInput and MLOutput for resource binding 14:54:36 ... I agree this is good for GPU Resources, but for CPU we lack means to do sync data conversion from an internal layout to a standard layout 14:55:35 ... for ArrayBuffer every time we need to download the data, major reason we want to introduce MLTensor to have a call to let the FW manage UL/DL, 14:56:10 ... this would mean the use case for Wasm with multiple graphs would be inefficient, with multiple data layout conversions happening 14:56:13 q? 14:56:22 q? 14:56:39 Ping: I hear what Chai says, and Ningxin's point 14:56:46 ... API should not be too complicated 14:57:00 ... the underlying format from what users can provide 14:57:20 ... w/o giving user high perf execution promise, defeats the purpose of this API 14:57:41 ... even for GPU, our execution could use different formats, vastly different from what users provide 14:58:06 ... the formats vary, having users provide the binding is performance issue, from API perspective cleaner 14:58:06 +q 14:58:50 ... usually from graph execution POV, seems to be good to have input and output, but now people chain a lot of small graphs and want to detect the boundary, ML is not one single graph anymore, multiple models interconnected 14:58:53 q? 14:59:11 Chai: just quickly, to move forward we need to tease a path for these two processes 14:59:26 ... 1) data UL/DL, independent of the layout format 14:59:48 ... 2) layout formats 15:00:20 ... native HW layouts exists and understand perf implications, requirement is not when data is copied but when it is converted 15:00:32 ... teasing these processes apart we can reach flexibility 15:01:18 anssik: Chai are you proposing to finish the device selection? 15:01:32 +1 15:01:36 Chai: separate issue for data UL/DL and another for layout conversion 15:01:42 ack ningxin_hu 15:01:46 ack chai 15:02:02 ningxin_hu: +1 Chai's resolution 15:02:42 proposed RESOLUTION: Open new issues for 1) data UL/DL 2) layout format conversions discussions 15:03:02 proposed RESOLUTION: Open new issues for 1) data UL/DL 2) layout format conversions discussions, and finish PR #162 addressing device selection 15:03:23 +1 15:03:30 +1 15:03:34 +! 15:03:36 +1 15:03:39 RESOLUTION: Open new issues for 1) data UL/DL 2) layout format conversions discussions, and finish PR #162 addressing device selection 15:03:46 RRSAgent, draft minutes 15:03:47 I have made the request to generate https://www.w3.org/2021/04/29-webmachinelearning-minutes.html anssik 15:04:38 Topic: Adjourn 15:04:41 RRSAgent, draft minutes 15:04:41 I have made the request to generate https://www.w3.org/2021/04/29-webmachinelearning-minutes.html anssik 15:09:43 [TAG guidance on device capabilities discovery & selection https://w3ctag.github.io/design-principles/#device-enumeration ] 15:18:07 s/conversions/conversion 15:18:09 RRSAgent, draft minutes 15:18:09 I have made the request to generate https://www.w3.org/2021/04/29-webmachinelearning-minutes.html anssik