Meeting minutes
<gabe_intel> hello!
Intersection between XLA & ONNX
anssik: deferring this topic since Paul is not attending today
ReLU and max WebNN API definitions
anssik: PR #62 merged
anssik: For the record, I'll defer my review to Chai and Rama when they're available to review, so consider my review optional
ningxin_hu: In this PR we have both high-level and low-level ops, ReLU is high-level and max is low level, created also a follow-up PR for LeakyReLU high-level op and min low level op
<JohnRochford> Self Intro: With W3C since 2013. Focus on accessibility. Developing ML model to simplify web text. Interested in AI bias.
LeakyReLU and min WebNN API definitions
anssik: Let's discuss the follow-up PR for leakyRelu and min:
<ningxin_hu> https://github.com/webmachinelearning/webnn/pull/63
Add leakyRelu and min (PR #63)
ningxin_hu: LeakyReLU is a variant of rectified linear function, this is a high-level op, links to its definition and how it decomposes into low level ops in the proposal
Variations of supported convolution type
anssik: Discuss two variants of the convolution operation we should consider:
… - Grouped convolution
… - Transposed convolution
Chai: This proposal came from the fact we try to spec a set of first-wave model ops, one is MobileNet
… at that time we decided to punt grouped conv, but turns out it is important
… Grouped convolution used largely, in 2-pass convolution in MobileNet
Chai: Tranposed convolution, used in autoencoder or models generating high resolution image
… Also known as "backward" convolution
… to compute convolution gradient during model training
Chai: 1D and 3D are different enough to warrant its own operators i.e. conv1d and conv3d
Chai: submitted a PR https://github.com/webmachinelearning/webnn/pull/64 2D pooling and reduce ops
anssik: any comments or feedback?
[none heard]
<Chai> please also review PR for pooling and reduction https://github.com/webmachinelearning/webnn/pull/64
ningxin_hu: for this issue, I know, as Chai mentioned, I raised open to support 1D and 3D convolution, but depending 1st-wave models we only require 2D conv that we already have
… as an issue with MobileNet, cannot be supported by current Conv2D op
… agreed to add grouped conv to existing conv2d to fix issue of MobileNet with respect to 1st-wave models
… then can decide whether to support transpose in next steps
Chai: transpose used in AlexNet, less urgent, 3D should prob be a separate op, TF also has it as a separate op
… Agree with Ningxin's proposal
ningxin_hu: if no objections, I can create a PR to add grouped conv support in existing op definition
<ping> does the group id makes it a generic conv op?
Ping: without the need to break them down into low-level primited
Chai: I think it makes sense to have grouped support in the same op, in transpose you'd need additional step before running conv
… 1D and 3D do not necessarily change the definition of ops, but need to accept 5D tensors for 3D case, can be thought of as a separate op
… feel like grouped conv is simple to implement
ping: my questions, should we make it more generic, w/o need to break it down into different dimentions
… should the API be high level?
… TF has multiple ops for that
… there's a new proposal in TF to combine all into one
Chai: the concern of having a really big op some op unable to handle some cases
… if there are too many features in one big op
… I work on DirectML so no problems there, but other APIs may not be able to do all these variants
PROPOSED RESOLUTION: Add grouped convolution support to existing conv2d definition in WebNN API
Chai: agree with proposal
Resolution: Add grouped convolution support to existing conv2d definition in WebNN API
Noise suppression use case
anssik: Belem submitted a PR:
[use case] Noise Suppression (PR #61)
anssik: To recap, our work and specifically op priorities in WebNN API are informed by use cases we've solicited in this group.
… the are documented in spec in prose and in a table that breaks them down by models
WebNN API first-wave models derived from the use cases
anssik: As you're probably aware so-called intelligent collaboration use cases have increased in importance during this time when people work from home
… We've committed to update the use cases to ensure they represent the demands of the users, and we think noise suppression is an important use case we should add to the WebNN API use cases:
anssik: it reads as follows: "A web-based video conferencing application records received audio streams, but usually the background noise is everywhere. The application leverages real-time noise suppression using Recurrent Neural Network such as [RNNoise] for suppressing background dynamic noise like baby cry or dog barking to improve audio experiences in video conferences."
anssik: any concerns in adding this use case for the WebNN API?
<Chai> I can take a look at the PR. The use case seems relevant.
Rafael: I'm fine adding noise suppression as a use case for the WebNN API
<ping> is noise cancellation supported at the OS level?
ping: does the web browser need this feature?
… if the noise cancellation is done on OS level do we need to do this in the web?
Chai: depends on what the model does, some models need FFT to do separation of content and noise
… if audio pipeline does that already, then fine, but if the model itself does transforms for performance reasons, then WebNN API needs to take care of that as well
Rafael: does adding this use case mean we need to add a bunch of new ops?
Chai: I suggest look at the proposed model rnnoise and go from these
ningxin_hu: one comment for noise cancellation usage, it is a long running task and it is very power sensitive, I know there's some hardware that helps with this usage, optimized for this use case
… I can look into specialized hardware optimized for these use cases
PROPOSED RESOLUTION: Adopt the noise suppression use case, evaluate rnnoise model for required ops
Resolution: Adopt the noise suppression use case, evaluate rnnoise model for required ops
Model Loader API
anssik: Last but not least, Jonathan will share with us the progress on the spec and prototype
Jonathan: We're interested and making progress, trying to figure out who at Google does the prototyping work
… discussed with Chrome GPU and Android NN and ChromeOS teams
… interest from developers in Google and 3rd party products to run the same ML models in native Android and web apps
… Android apps developed with TF Lite models
… ChromeOS team is interested in supporting TF Lite models (too?)
… to get started, they're interested in Model Loader API, OK with the fact TF Lite model format won't be a "standard" model format, interested to contribute engineering resources for TF Lite Model Loader API PoC
Jonathan: starting the coming weeks, we have an engineer from CrOS team working with GPU team to prototype Model Loader API as we have in the GH repo
… any issues to be reported in the spec repo
… we want to connect the effort with WebNN API, align in the longer term between the two APIs being worked on in this group
… like to get conversation going, want to have engineers working on PoC come to this group to introduce the plan and connect with this group
… the intention is not to proposed TF Lite as "the standard format", that's just for PoC purposes
… open to explore how to make this support ONNX and other model formats
Rafael: we at Microsoft, also have a plan to prototype Model Loader API
… we should collaborate and join forces on this PoC exploration
… also partner interest
ningxin_hu: very good plan! thanks for sharing
… Jonathan mentioned long-term goal to align builder and loader APIs
… there's some conversation how to align these re input and output, execution
… if you're interested, we can open an issue to discuss this alignment in this group, in an open space
Chai: I agree this is a good effort, once we know what models we want to do PoC for, we should add them to the set of models WebNN supports
<ningxin_hu> https://github.com/webmachinelearning/webnn/blob/master/op_compatibility/first_wave_models.md
ningxin_hu: want to echo Chai's point, for first-wave models table we list the models in two formats, one in ONNX one in TF Lite
… if Google prototypes Model Loader API please review this table
… maybe we can define next-wave models
<ningxin_hu> https://github.com/webmachinelearning/webnn/pull/52