Meeting minutes
Announcements
anssik: 2020-02-13 update: Web & ML workshop postponed sine die due to ongoing challenges around international travel
… all registrants have been informed, we do not yet have a new date, monitoring the situation actively
… workshop page also updated
[POSTPONED] W3C Workshop on Web & Machine Learning
anssik: questions?
conv2d op definition
anssik: nignxin submitted a PR https://github.com/webmachinelearning/webnn/pull/43
… signature only, to be evolved in subsequent PRs to add more details, proceducal algorithms
… open is the default tensor layout: NHWC or NCHW
… NHWC matches HTMLMediaElements
… NCHW more natural to convolution
… perf implications: NCHW more GPU-friendly, but situation evolves
… anssik: what are the problems we want to solve? are they really problems we need to solve on the WebNN API level?
ningxin_hu: PR submitted, based on original proposal from Nikhil and compat study we put together in this group
… no issues around arguments themselves, the default input tensor layout is he only open issue
… NHWC per Nikhil's proposal, HTMLMediaElements uses this layout as the default
Chai: agree with Ningxin, the most important change we should make is to make sure conv2d support both the layouts, since it depends how the model is trained which layout to use
… most training scripts use NCHW thanks to cuDNN
… when training on GPU this is a reasonable default
… ONNX and CoreML default to NCHW layout
… both the formats want to perform well on GPU
… side question about HTMLMediaElement, maybe not so relevant since need to convert to tensor anyway
… prefer to default to NCHW
ningxin_hu: I updated Conv2d compat table, all native APIs support NCHW
Rafael: the PR seems fine, but eventually we need to allow ppl to pass image elements into these APIs
Chai: propose we create a new issue for passing image elements into conv2d
PROPOSED RESOLUTION: Add data format argument to conv2d to allow input tensor layout to be configured as NHWC or NCHW, default to NCHW
anssik: any concerns with the proposed resolution?
PROPOSED RESOLUTION: Add data format argument to conv2d to allow input tensor layout to be configured as NHWC or NCHW, the API needs to support both, default to NCHW
Resolution: Add data format argument to conv2d to allow input tensor layout to be configured as NHWC or NCHW, the API needs to support both, default to NCHW
Remove Compilation.finish and setPreference
anssik: this PR https://github.com/webmachinelearning/webnn/issues/39 spun off from float16 OperandType PR #35
… based on feedback new refined IDL proposed by Ningxin: https://github.com/webmachinelearning/webnn/issues/39#issuecomment-588865620
PROPOSED RESOLUTION: Refactor into createCompilation(CompilationPreference preference) and createExecution()
Rafael: is Execution an object to bind inputs to?
ningxin_hu: yes, that's the current design, also output bound to it
Rafael: Execution is a one-shot?
ningxin_hu: right
Rafael: why execution needs to return a promise, and not create a new?
… I understood compilation can take a long time
… why execution also needs to return a promise and not return (synchronously)
[scribe missed]
Rafael: fine with the proposed IDL, can maybe refine in fugure updates these API further
Rama: createExecution needs to wait first, motivation for the design?
… buffers may need to wait perhaps?
ningxin_hu: in prototyping noticed lag in first execution due to buffer allocation
PROPOSED RESOLUTION: Refactor into createCompilation(CompilationPreference preference) and createExecution()
anssik: any concerns?
<Rafael> +1 proposed IDL looks good to me.
Resolution: Refactor into createCompilation(CompilationPreference preference) and createExecution()
Handling unsupported OperandType
anssik: this issue was actually closed, since the real issue is with quantization-specific params https://github.com/webmachinelearning/webnn/issues/44
ningxin_hu: https://github.com/webmachinelearning/webnn/issues/36 may need to be reopened
Chai: issues are a bit unrelated, want to keep OperandType and OperandDescriptor clean for the future
… quantization is still evolving, now APIs implement linear quant, if we add quant option in enum type and descriptor, can be potentially ongoing
… proposal to keep operand just about datatype
… I can add more detail in the issue #44
ningxin_hu: #44 is clear, my question is float16 was just added as a datatype but if not supported by underlying platform, how the API should react?
Chai: the API should fail in such case
ningxin_hu: we need to specify how to surface this error condition through the API
… prefer to reopen #36 to flesh out how to signal the error in it
WebML synopsis at WebGPU F2F
anssik: I wanted to discuss key WebML takeaways from WebGPU F2F
WebGPU CG F2F 2020-02-13 meeting minutes
Rafael: I can give a summary of summary :-)
… Apple seemed rather supportive of a load model API
… the two groups agreed to work together
Chai: on Q from the F2F, what does WebML want from WebGPU? Custom op support discussed, how to make interop happen, exercise for the future to PoC ML custom op in terms of WebGPU, how custom op would look like
<ningxin_hu> https://github.com/webmachinelearning/webnn/issues/6
ningxin_hu: re custom ops WebGPU, we have issue #6 about this topic and have done some preliminary investigation into the topic, using WebGPU compute shader
Chai: exercise was to prototype that but in JS, not internally calling into OS API but on a higher-level, figure out how buffer sharing would work, WebGPU has a BufferResource, how graph builder API would consume it
Inference API to load and run a model
anssi: Let's discuss feedback and decide next steps for inference API to load and run a model.
Revisit inference API to load and run a model #41
jonathan: thanks to people who commented ningxin and chai
… discussed whether this is complementary to the existing API, consensus seems to be it is
… another discussion, whether we need separate compilation step or not, it seems the answer is probably yes
… discussions around what information needs to be exposed to web developer vs graph builder API
… also discussed what workstreams could come out of this
… ningxin_hu's existing POC work would be extended with model loader API, possibly
… personally would like to be able to separate the two API surfaces, if an implementer wants to implement only one, not both
////////
https://webmachinelearning.github.io/charter/
quote from the charter: "Allow to construct a neural network computational graph by common building blocks, including constant values and base operations such as convolution, pooling, softmax, normalization, fully connected, activation, recurrent neural network (RNN) and long short-term memory (LSTM);"
Chai: I feel like it is important to define the graph builder API, since with load model API we'd abstract out many important details
… model loader API would be like the OS-provided API
Rafael: agree with Chai, just to add, if the group cannot agree on a model format to digest, then graph builder API is a great hedge and allows leaving model parsing etc. to JS
ningxin_hu: would like to hear feedback on our POC proposal
ningxin_hu: The existing WebNN Proof-Of-Concept (POC) is based on Chromium and implements the model-builder API on top of Android NNAPI, Windows DirectML and macOS MPS/BNNS. For the new model-loader API prototype, I propose to extend the existing Chromium-based POC and start with WinML.
https://github.com/webmachinelearning/webnn/issues/41#issuecomment-588849050
https://github.com/otcshare/chromium-src
Chai: in short, I think if we take the load model API separate from graph builder API, could have two parallel tracks that evolve differently, maybe not so good idea since we do not want to punt important design issues to OS
… load model API as a convenience on top of graph builder API is my preference
anssik: anyone else interested in prototyping the load model API?
(hearing none)