14:47:49 <RRSAgent> RRSAgent has joined #webmachinelearning
14:47:49 <RRSAgent> logging to https://www.w3.org/2020/03/05-webmachinelearning-irc
14:47:54 <Zakim> Zakim has joined #webmachinelearning
14:48:08 <anssik> RRSAgent, make logs public
14:48:10 <anssik> Meeting: WebML CG Teleconference – 5 March 2020
14:48:15 <anssik> Chair: Anssi
14:48:20 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2020-03-05-agenda.md
14:48:26 <anssik> Scribe: Anssi
14:48:37 <anssik> scribeNick: anssik
14:48:40 <anssik> Present+ Anssi_Kostiainen
14:48:57 <anssik> RRSAgent, draft minutes v2
14:48:57 <RRSAgent> I have made the request to generate https://www.w3.org/2020/03/05-webmachinelearning-minutes.html anssik
14:56:35 <Rafael> Rafael has joined #webmachinelearning
14:58:44 <anssik> Present+ Rafael_Cintron
15:00:21 <anssik> Present+ Ningxin_Hu
15:00:31 <Rama> Rama has joined #webmachinelearning
15:00:47 <ningxin_hu> ningxin_hu has joined #webmachinelearning
15:01:03 <anssik> Present+ Ganesan_Ramalingam
15:01:09 <ningxin_hu> Present+ Ningxin_Hu
15:02:44 <Jonathan> Jonathan has joined #webmachinelearning
15:03:02 <anssik> Present+ Chai_Chaoweeraprasit
15:03:13 <anssik> Present+ Jonathan_Bingham
15:03:43 <anssik> TOPIC: conv2d op definition
15:03:46 <daniel_smilkov> daniel_smilkov has joined #webmachinelearning
15:03:49 <anssik> PR reviewed and ready to land
15:03:55 <anssik> Present+ Daniel_Smilkov
15:04:05 <anssik> -> https://github.com/webmachinelearning/webnn/pull/43 Add conv2d (PR #43)
15:04:10 <anssik> PROPOSED RESOLUTION: Merge PR#43 to add conv2d definition to the spec
15:04:13 <Chai> Chai has joined #webmachinelearning
15:05:18 <anssik> ningxin_hu: I see daniel and nikhil online, want to check the signature looks good
15:06:06 <anssik> HTML preview of conv2d: https://pr-preview.s3.amazonaws.com/huningxin/webnn/pull/43.html#api-neuralnetworkcontext-conv2d
15:06:59 <anssik> Nikhil: we haven't had time yet to look at this
15:07:09 <anssik> anssik: ok to merge and evolve in subsequent PRs?
15:07:16 <anssik> Nikhil: sounds good
15:07:42 <anssik> Daniel: the same, looks good, will review the details later
15:07:55 <anssik> RESOLUTION: Merge PR#43 to add conv2d definition to the spec
15:08:04 <anssik> TOPIC: matMul op definition
15:08:13 <anssik> -> https://github.com/webmachinelearning/webnn/issues/27 [op compatibility] matMul (issue #27)
15:08:27 <anssik> -> https://github.com/webmachinelearning/webnn/blob/master/op_compatibility/matmul.md matMul compat table
15:10:05 <anssik> ningxin_hu: the compat table for matMul op in GitHub in a familiar location at https://github.com/webmachinelearning/webnn/blob/master/op_compatibility/matmul.md
15:10:16 <anssik> ... Chai helped with DirectML mapping
15:11:12 <anssik> ... NNAPI does not have matMul defined, so looked at TF Lite matMul, references found in the doc
15:12:31 <anssik> [discussing details of dimensionality]
15:13:23 <anssik> ningxin_hu: NNAPI open question: NNAPI: for N-D output when N >2, is it correct to map TF-Lite Pack op to NNAPI ANEURALNETWORKS_RESHAPE and ANEURALNETWORKS_CONCATENATION ops?
15:14:25 <Chai> q+
15:15:03 <anssik> anssik: can Nikhil find someone from Google to help with NNAPI specifics?
15:15:09 <anssik> Nikhil: can look for a contact
15:15:15 <anssik> ack Chai
15:16:04 <anssik> Chai: ningxin starts from numpy, but not sure if that maps nicely to all these APIs
15:17:05 <anssik> ... to multiply matrices, DNNL takes multiply-add approach
15:17:29 <anssik> ... question to all, whether this would be considered instead or in addition
15:18:19 <anssik> ... also does this group want single matMul that can handle N-dim tensors, or do genetic matMul and generic Gem
15:18:46 <anssik> s/would be/should be
15:19:59 <anssik> ningxin_hu: Chai's right re DNNL, but want to hear Nikhil's feedback
15:20:16 <anssik> Daniel: the reason for not fusing things on the op level is we want to reduce the level of ops in the spec
15:20:27 <anssik> ... if we allow fusion capability we might have more ops than we want
15:20:41 <anssik> ... we want ops to do less things, and let compiler fuse things under the hood
15:20:41 <ningxin_hu> s/gem/gemm/
15:21:16 <anssik> ... if we allow fusing on a logical level, we're leaking hw optimizations
15:21:25 <anssik> should keep bias as its own op
15:21:37 <anssik> .../should keep bias as its own op/... should keep bias as its own op
15:21:46 <Chai> q+
15:22:30 <anssik> ack Chai
15:22:54 <anssik> Chai: agree with Daniel's philosophy, a great topic
15:23:19 <anssik> ... the context in which the compiler is able to use auto fusion, if not careful, hard to discover
15:23:44 <anssik> ... say you have a graph that has multiply-add as to different ops, most hw able to fuse this bias piggybacking on it
15:23:56 <anssik> ... question is how does the sw level know this context can be fused
15:24:12 <anssik> ... in this struggle one idea in ONNX is called ONNX function
15:25:16 <anssik> ... in that you define the whole op as a multiple-add function(?), then if the hw is not able to support mad that gets broken down to two ops it is able to support
15:25:45 <anssik> ... wanted to make a data point, a great topic, everyone trying to address this problem on a native level
15:26:01 <anssik> Nikhil: MLIR approach takes the function approach, pushing it entirely to the compiler
15:26:45 <anssik> ... when downleveled, there are two components, folding weights and doing fusing, happens in the compiler entirely
15:26:56 <anssik> ... I want to also throw another thing out there
15:27:02 <anssik> ... something we've been thinking about at Google
15:27:45 <anssik> ... op-level is the CG's current approach, another project we should pay attention to is XLA
15:28:09 <anssik> ... lower level than ops, higher than hardware
15:28:18 <anssik> ... designed to GPUs, has around 80 ops
15:28:45 <anssik> ... people on XLA team say they can run every NN with this op set
15:29:17 <anssik> ... a lot of backends for XLA
15:29:39 <anssik> ... currently a Google project: https://www.tensorflow.org/xla/architecture
15:30:35 <daniel_smilkov> https://www.tensorflow.org/xla/architecture
15:30:43 <daniel_smilkov> https://www.tensorflow.org/xla/operation_semantics
15:31:36 <anssik> Daniel: the primary motivation we're starting to be interested in XLA HLO, it is trying to remove reliance on custom ops
15:31:49 <anssik> ... something around that level can likely stand the test of time
15:32:00 <anssik> ... these ~80 ops haven't grown
15:32:15 <anssik> ... much more stable than ONNX or TF Lite ops that grow faster
15:32:39 <anssik> Nikhil: also want to bring some caveats of XLA HLO
15:32:54 <anssik> ... the binary format not backwards compatible
15:33:01 <anssik> ... this is clearly an issue for the Web
15:34:27 <anssik> Chai: I believe, issue #41 has extensive discussion on the topic, XLA also discussed in it, see https://github.com/webmachinelearning/webnn/issues/41
15:35:51 <anssik> ningxin: back to compat study findings
15:36:42 <anssik> ... remaining opens in addition to NNAPI are MPS and BNNS
15:37:21 <anssik> ... in MPS support matMul through FullyConnected
15:37:56 <anssik> ... also MPSNDArrayMatrixMultiplication can handle matMul on MPS
15:38:14 <anssik> ... help from Apple requested
15:39:19 <anssik> anssik: any other comments or feedback?
15:39:21 <anssik> [none]
15:39:33 <anssik> TOPIC: Handling unsupported OperandType
15:39:40 <anssik> -> https://github.com/webmachinelearning/webnn/issues/36 Handling unsupported OperandType (issue #36)
15:39:47 <anssik> anssik: Discuss how to surface "datatype not supported by underlying platform" errors through the API
15:41:14 <anssik> ningxin_hu: initial idea is to throw exception at compile time
15:41:35 <anssik> Chai: I have another proposal, I believe in this case it should be error code instead of an exception
15:41:43 <anssik> ... feedback provided on the issue
15:42:06 <anssik> ... essentially, you don't want everything to be exception, only unrecoverable errors
15:42:19 <anssik> ... everything errors should be signaled using error codes
15:42:56 <anssik> ... the caller should fall back to compile the model with different options, that is recoverable error
15:43:07 <anssik> ... do we need an error code, or simply return null?
15:43:20 <anssik> ... no preference there, but prefer not to have it as an exception
15:43:55 <anssik> Rafael: I agree with Chai, exceptions should be thrown when develop has done something totally wrong, in this case this is not the case
15:44:45 <anssik> anssik: any prior examples from WebGL or WebGPU?
15:45:01 <anssik> Rafael: yes, there's a bunch of those
15:46:54 <Rafael> https://www.khronos.org/registry/webgl/specs/latest/1.0/#5.2.1.  You need to scroll down a little bit before you get to the failIfMajorPerformanceCaveat context attribute.
15:47:12 <anssik> Chai: I'll look at WebGL examples and see how to adopt those patterns in WebNN
15:48:04 <anssik> TOPIC: Inference API to load and run a model
15:48:27 <anssik> -> https://github.com/webmachinelearning/webnn/issues/41 Revisit inference API to load and run a model (issue #41)
15:48:27 <anssik> -> https://github.com/jbingham/web-ml-inference
15:49:21 <anssik> anssik: Final call for feedback prior to adding this API into the group's scope
15:53:17 <zkis> zkis has joined #webmachinelearning
15:53:56 <anssik> Rafael: do we need to change the charter to take on this work?
15:55:16 <anssik> anssik: not necessarily, I'll look into this and try to see if we can add this new proposal in scope without changing the charter
15:55:56 <anssik> anssik: does someone thing this new proposal should NOT be incubate further?
15:56:07 <anssik> [hearing no opposition]
15:56:27 <anssik> Rafael: Microsoft would like to see load model API incubated in this CG
15:57:16 <anssik> Jonathan: Google is in support of this proposal (obviously :-))
15:59:22 <anssik> Jonathan: about the charter, if we want to update it, based on comments from Microsoft, make a draft and not submit it before review
16:01:36 <anssik> Rafael: that's how other CGs do it
16:03:33 <anssik> TOPIC: Adjourn
16:03:40 <anssik> RRSAgent, draft minutes v2
16:03:40 <RRSAgent> I have made the request to generate https://www.w3.org/2020/03/05-webmachinelearning-minutes.html anssik
16:04:11 <anssik> Present+ Paul_McDaniel
16:04:18 <anssik> RRSAgent, draft minutes v2
16:04:18 <RRSAgent> I have made the request to generate https://www.w3.org/2020/03/05-webmachinelearning-minutes.html anssik
18:29:08 <Zakim> Zakim has left #webmachinelearning
18:47:20 <zkis> zkis has joined #webmachinelearning
19:36:22 <zkis> zkis has joined #webmachinelearning