15:02:01 RRSAgent has joined #webmachinelearning 15:02:01 logging to https://www.w3.org/2021/03/04-webmachinelearning-irc 15:02:04 RRSAgent, make logs Public 15:02:05 please title this meeting ("meeting: ..."), anssik 15:02:06 Chair: Anssi 15:02:19 Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2021-03-04-agenda.md 15:02:25 Scribe: Anssi 15:02:29 scribeNick: anssik 15:02:33 Present+ Anssi_Kostiainen 15:02:40 Present+ Ganesan_Ramalingam 15:02:47 Present+ Chai_Chaoweeraprasit 15:02:56 Present+ Ningxin_Hu 15:03:01 Present+ Zoltan_Kis 15:03:06 ningxin_hu_ has joined #webmachinelearning 15:03:29 RRSAgent, draft minutes v2 15:03:29 I have made the request to generate https://www.w3.org/2021/03/04-webmachinelearning-minutes.html anssik 15:03:44 Topic: TAG review feedback - open PRs 15:04:00 anssik: couple of TAG review issues have PRs in review, let's discuss those. 15:04:01 Subtopic: [tag-tracker] NamedOutput mechanism clarification #140 15:04:05 Present+ Rafael_Cintron 15:04:16 -> https://github.com/webmachinelearning/webnn/issues/140 NamedOutput mechanism clarification #140 15:04:18 Rama has joined #webmachinelearning 15:04:20 -> https://github.com/webmachinelearning/webnn/pull/147 PR #147 15:04:36 RafaelCintron has joined #webmachinelearning 15:04:38 anssik: PR #147 seems to be pending review from Ping and Chai. Any initial reactions? 15:06:35 ningxin_hu: this PR tries to use W3C convention to describe the algorithm similar to other web specs 15:06:58 ping_yu has joined #webmachinelearning 15:08:25 ... for compute method, interacting with the device, I read the WebGPU spec and there are many methods that are similar and use a timeline concept, so though reuse that concept in this concept, to help spec writing and also easier to read by people familiar with WebGPU spec conventions 15:08:48 ... no feedback yet from WebGPU participants, looking forward to that feedback. 15:09:11 Present+ Ping_Yu 15:09:45 I do not have immediate feedback but will review this week. 15:10:06 will do 15:10:38 Subtopic: [tag-tracker] WebGL and WebGPU interops (related issues #135, #141, #136) 15:10:47 -> https://github.com/webmachinelearning/webnn/issues/135 [tag-tracker] Create context from external sources #135 15:10:52 -> https://github.com/webmachinelearning/webnn/issues/141 [tag-tracker] Prefix generic interface names #141 15:10:56 -> https://github.com/webmachinelearning/webnn/issues/136 [tag-tracker] Switch to use constructor #136 15:11:01 anssik: issues #135, #141, #136 are addressed by PR #149 15:11:05 -> https://github.com/webmachinelearning/webnn/pull/149 PR #149 15:11:10 anssik: Chai's PR #149 comment has a summary: 15:11:21 ... - Create context from external sources e.g. WebGLRenderingContext and WebGPU device (#135) 15:11:27 ... - Make power preference part of context creation options. 15:11:32 ... - Constant operands can be created from either WebGL or WebGPU buffers 15:11:36 ... - Model inputs and outputs can be bound with WebGL or WebGPU textures 15:11:39 ... - Prefix all types with "ML". Simplify "NeuralNetworkContext" to just "MLContext" (#141) 15:11:43 ... - Switch to use constructor for MLModelBuilder instead of factory method (#136) 15:12:05 Chai: purpose as the name says, provide interop with WebGL and WebGPU, also zerocopy interp efficiently 15:12:19 ... TAG reviewer mentioned that the way we deal or not deal with device might be a problem 15:12:34 ... up to this point we haven't exposed the device concept 15:13:12 ... we can interop with a device even without such a notion, the first aspect people should look at is the way how NeuralNetworkContext is created, it is called MLContext now 15:14:11 ... we can create it with WebGLRenderingContext of WebGPUDevice, chose Device over Adaptor since WebGPU has a way to choose an adapter, and choosing an adapter is pretty important, e.g. when you have multiple adapters in your system, it is either up to the framework or the web developer to choose the right one 15:14:40 ... we also want to create out context the way we wanted, looked at the current API with compilation power pref, we need to frontload that when the context is created 15:15:16 ... so made it into MLContextOptions, web developers now need to deal with the device earlier in the flow, the benefit is this is consistent with the way we interop with the device 15:15:29 ... if we create the context outselves, the model is inconsistent with the device 15:15:57 ... that's a minor change introduced by this PR, the rest is just common sense, you can create a context from a device and bind it 15:16:21 ... the input to the model can be WebGL or WebGPU textures in many cases due to visual input use cases 15:16:42 ... we can use WebGPUBuffers as input, per Ningxin's suggestions, that sounds good to me 15:17:20 ... about the model constants, limited to buffers, all the GPU vendors deal with buffers as constants, conv weights etc. no GPU vendors support textures due to its complexity 15:17:33 ... all the constants in the model supported by buffers 15:18:01 ... we used to use factor, not use ctor instead since it is simpler and aligns with API Design Guidelines 15:18:07 q+ 15:18:39 ack RafaelCintron 15:18:52 RafaelCintron: in general I'm supported of Chai's PR #149 15:19:19 ... we do not need to solve this in this PR, we should look at how the promises in WebNN API interact with these other APIs we add dependency on to 15:19:54 ... while the promise is off doing its thing, if WebGL or WebGPU buffers are changes, might introduce a race condition 15:20:27 ... a web developer expects a more deterministic API perhaps 15:21:05 ... we may have to modify this a bit, it may not be as seamless as we think to go back and forth with WebNN and WebGL/GPU buffers 15:21:41 Chai: great idea, maybe create an issue for this Rafael? 15:22:17 q? 15:22:40 Chai: PR is pretty loaded, so can address Rafael's comments separately 15:23:25 q+ 15:23:45 ack ningxin_hu_ 15:24:34 ningxin_hu_: according to my understanding the MLModel is device agnostic? 15:24:55 ... do we still need to keep the MLModel device agnostic, or leave that for compile time? 15:25:12 ... for example, power preference was prior defined at compile time 15:25:46 ... web developer has to build a model in a device agnostic way and compile for different devices or power preferences 15:26:15 Chai: whether the model should be device agnostic is an interesting one 15:26:36 ... when you create a model builder, which is consistent with WebGPU, device is set up as the first thing 15:26:47 ... in this approach, the model cannot be completely out of context of the device 15:26:55 ... because device is specified in the beginning 15:27:20 ... can think that at compile step, if already equipped with a device, it is essentially how to fuse or how to do layout assignments 15:27:40 ... not separating out device agnostic phase from compilation that can always be device specific 15:27:48 model topology should be device agnostic, compilation should be device specific. 15:28:25 ... the model may not be agnostic to the device, maybe the caller can use WebNN to build a graph and then serialize it out 15:28:51 ... you can do this for MLModel, when frontloading the device it is hard to mentally think of objects coming later be device agnostic 15:29:08 ... can spec that "MLModel is device agnostic" but in practice may not be 15:29:19 ningxin_hu_: do we need to put the device in the compile API instead? 15:29:42 Chai: the issue is you may want to construct a model by giving it device specific constants 15:30:20 ... imagine, build a model off of WebGLContext, you'd have to do MLContext.constant, so dealing with a device resource even at the time you construct the model 15:30:51 ... if you interop with an external device, these constants are already device resources 15:31:14 ningxin_hu_: it makes sense if like in your PR constants can be created from WebGL/GPU buffers 15:31:23 where constant is should not prevent creating device agnostic model 15:31:37 ... I know interop for execute and compute of the model, interacts with device buffer is useful 15:31:44 q? 15:32:07 ... I don't understand model creation with device resource, what's the use case for that? 15:32:32 I have some comments 15:32:40 q+ 15:32:43 q? 15:33:36 Chai: there are cases when we interop with scenario where the data is on device as device resource, in those cases WebGL uses upload to GPU and then construct the graph 15:33:52 ... there are cases where you construct the graph when all the weights have already been uploaded to the GPU 15:34:20 ... agree, the first think to do is to construct the model, but also cases esp for interop when weights are already in GPU buffer 15:34:35 ... and don't want to do a round-trip 15:34:37 q? 15:34:57 ningxin_hu_: thanks for the explation 15:34:59 ack ping_yu 15:35:33 Ping: I like this conversation and align with Ningxin's proposal, model should be device agnostic 15:36:01 ... example from TF.js, you do not know if constant is on the GPU, from user's POV 15:36:20 ... in our framework we do the data transfer for them, we will not prevent that 15:36:39 ... I feel the model should be device agnostic, execution device specific 15:36:54 RRSAgent, draft minutes v2 15:36:54 I have made the request to generate https://www.w3.org/2021/03/04-webmachinelearning-minutes.html anssik 15:37:17 Meeting: WebML CG Teleconference – 4 March 2021 15:37:21 RRSAgent, draft minutes v2 15:37:21 I have made the request to generate https://www.w3.org/2021/03/04-webmachinelearning-minutes.html anssik 15:38:56 Chai: quick comment, as I mentioned, it is a design tradeoff, if we want model to be device agnostic, all device interop must be done later after the model has been constructed 15:39:19 ... if we want to allow max interop when model weights are already uploaded 15:40:40 ... I'll think this a bit, my hunch you either support the proposed interop proposal, or you don't, cannot revert and do it differently 15:41:29 Subtopic: [tag-tracker] Explainer feedback 15:41:33 15:41:37 -> https://github.com/webmachinelearning/webnn/issues/146 [tag-tracker] Explainer update per TAG review feedback #146 15:41:56 -> https://github.com/webmachinelearning/webnn/pull/148 PR #148 15:42:08 anssik: in summary, PR #148 notes use cases up front, clarifies AI & ML terminology usage, adds a high-level comparison table for explaining the positioning of WebNN API vs. Model Loader API. 15:43:02 15:43:04 ... any comments, OK to merge? 15:43:44 Topic: TAG review feedback - issues with active discussion 15:43:44 yes looks good. i've signed off on it 15:44:00 anssik: The ask was to take a look at these issues and make sure that you provide your feedback in comments. The editors will take your feedback into considerations when crafting PRs for these issues. 15:44:09 ... let's peek into these one by one to get reactions 15:44:15 Subtopic: [tag-tracker] Isomorphic JS story, worker scope exposure? 15:44:23 -> https://github.com/webmachinelearning/webnn/issues/142 [tag-tracker] Isomorphic JS story, worker scope exposure? #142 15:44:36 anssik: In response to Rafael's comment on Node and WebGL exposure, Sangwhan made a point a Foreign Function Interface (FFI) bindings to a compute backend such as CUDA would be feasible. 15:45:12 ... Ningxin points our webnn-polyfill CPU backend already works in Node.js for testing purposes. Also, the Node.js binding to the proposed webnn-native should also work. 15:45:19 ... the discussion seems to be pending Sangwhan's further comments? 15:45:25 ... any thoughts from folks on the call? 15:46:18 q? 15:46:33 Subtopic: [tag-tracker] Prefix generic interface names? 15:46:38 -> https://github.com/webmachinelearning/webnn/issues/141 [tag-tracker] Prefix generic interface names? 15:46:48 anssik: the feedback from the TAG was: "A lot of the names are very generic (Operand, Compilation) - this feels like something we might want to prefix with something or synchronize with TC39 about" 15:46:54 ... and we actually have a PR for this issue thanks to Chai: 15:46:58 -> https://github.com/webmachinelearning/webnn/pull/149 PR #149 15:47:17 anssik: Chai want to walk us through the PR? 15:47:55 Chai: I simply take a cue from ML attribute and use that as a prefix for consistency, Operand -> MLOperand, NeuralNetworkContext -> MLContext 15:48:08 ... when I look at WebGPU specs, they use GPU as a prefix 15:48:19 ... same with WebGL, prefixed with WebGL 15:48:59 anssik: any comments re naming? 15:49:09 Subtopic: [tag-tracker] Ergonomics of the JS examples 15:49:14 -> https://github.com/webmachinelearning/webnn/issues/139 Ergonomics of the JS examples #139 15:49:29 anssik: TAG feedback was: "While the limitations of JavaScript probably contribute a lot to this, but the ergonomics of this API based on example code might have room for improvement." 15:49:53 ... Sangwhan adds some more details in comments: "unless the compute graph's topology is complicated (see: Inception V3...) reading the code for a sequential model should give you a rough idea of what the model is doing, but right now this is difficult." 15:50:06 ... I think a key design choice the group took early on was to position the API with a JS framework as its key consumer 15:50:20 anssik: thought on ergonomics? 15:50:59 q+ 15:51:08 ... to address this issue, we could note in the spec the key consumer for the API is JS frameworks 15:51:12 ack zkis 15:51:42 zkis: just a quick question, is it necessary to expose NeuralNetworkContext, now that we have ctor for creating the model? 15:51:57 ... reason for multiple contexts? 15:52:12 Chai: quickly, the notion of a context is basically to keep the global state 15:52:33 ... if you think about the implementation of this API in the browser, they need to keep the notion of a device in case of GPU 15:52:45 ... otherwise lifetime is not very clear 15:52:59 ... low-level APIs come with a notion of context 15:53:26 zkis: is it possible to have an internal slot that developers don't use directly? Context could be internal 15:54:32 Chai: for flexibility, the developer may choose that this specific model I want to use this specific device, if we don't want to give people that option, then the ML attribute could be its own context, depends how much we want people to interop with WebNN and WebGL/GPU APIs 15:54:56 zkis: earlier we had a similar problem, e.g. WebNFC with multiple adapters 15:55:23 ... before context owned by model builder, now it is moved our as a ctor so we have a default context 15:57:13 Subtopic: [tag-tracker] String enum for activations 15:57:19 -> https://github.com/webmachinelearning/webnn/issues/138 String enum for activations #138 15:57:39 anssik: TAG says: "If there are layers that will be taking activations as string enums, there should simply be a string enum for activations rather than have it just in RecurrentNetworkActivation. (One may argue that hyperbolic tangent is RNN specific, but..." 15:57:50 ... perhaps we could note this detail in the spec as an informative note for readers? 15:58:09 anssik: comments? 15:58:20 Subtopic: [tag-tracker] Clarify which view/reshape like function are expected to copy 15:58:32 -> https://github.com/webmachinelearning/webnn/issues/137 [tag-tracker] Clarify which view/reshape like function are expected to copy #137 15:58:37 anssik: TAG said: "I see quite a few view/reshape like functions, which of these are expected to copy and which are not? Probably good to note this in the spec." 15:59:12 ... Sangwhan asks that wouldn't it make sense to define a common term for logical tensor changes (e.g. views?) somewhere early in the document so that concept can be re-used? 15:59:22 ... in response to a review of PR #144 that proposes to address this issue #137 16:00:10 Chai: I did PR #144, I added some clarification on which copy 16:00:33 https://github.com/webmachinelearning/webnn/pull/144/files/44870dee0271e50ead69da93bf4ef0b9c206537e 16:00:49 Chai: will take a look at this comment 16:02:30 Topic: web-native update 16:02:44 An update on webnn-native open-source project progress and plans deferred for next call 16:02:49 -> https://github.com/webmachinelearning/webnn-native/pull/1 webnn-native PR #1 initial implementation 16:02:55 anssik: Dawn license comments to be addressed before landing 16:03:10 sounds good 16:04:21 Topic: Adjourn 16:04:47 s/deferred for/deferred to 16:05:04 RRSAgent, draft minutes v2 16:05:04 I have made the request to generate https://www.w3.org/2021/03/04-webmachinelearning-minutes.html anssik 16:14:30 s/license/Copyright statement 16:14:32 RRSAgent, draft minutes v2 16:14:32 I have made the request to generate https://www.w3.org/2021/03/04-webmachinelearning-minutes.html anssik 17:47:45 zkis has joined #webmachinelearning 17:57:26 Zakim has left #webmachinelearning