W3C

– DRAFT –
WebML CG Teleconference – 25 June 2020

25 June 2020

Attendees

Present
Anssi_Kostiainen, Chai_Chaoweeraprasit, Jonathan_Bingham, Ningxin_Hu, Paul_McDaniel, Ping_Yu, Rafael_Cintron, Wonsuk_Lee
Regrets
-
Chair
Anssi
Scribe
Anssi, anssik

Meeting minutes

Set inputs and outputs by string key

Set inputs and outputs by string key (PR #69)

anssik: PR reviewed and merged. Nice API ergonomics improvement
… fixes issue to allow a graph to declare the keys for its inputs and outputs
… string-based keys are developer friendly, easier to memorize, also aling with the Model Loader API
… comments?

ningxin_hu: thanks for the review all, this issue was raised by Chai and Rafael
… this PR fixes the issue #67
… two issues in the previous API: 1) we use integer-based index for inputs and outputs, in our latest builder API indexing of inputs is very unclear, implicitly set
… 2) Rafael raised a point we need to align with the Model Loader API that uses string-based keys to set the model exec inputs and outputs
… as mentioned last time, we created this issue and moved the conversation to GH and based on that I created the PR and tried to fix both the issues at once
… if you look at the Model Loader API the current shape is not exactly the same, it uses some dictionary directly in the predict API
… maybe Model Loader API needs some changes a bit to align with the WebNN API convention

anssik: in general support the plan to align the two APIs in this respect

[Ningxin recaps the new feature introduced in PR #69 to Jonathan]

Jonathan: agree good idea to unify the API for that feature

Rafael: good idea to allow inputs and outputs specified as strings

Ping: what are the string values for those names?

ningxin_hu: for WebNN API, it's developer-friendly name to identify inputs and outputs
… for Model Loader API, ONNX uses strings for their inputs and outputs

https://‌webmachinelearning.github.io/‌webnn/#examples

ningxin_hu: in my current polyfill, these strings are translated into internal id, e.g. integer-based keys

Ping: on the model side, that needs to be maintained inside the model architecture?

ningxin_hu: looking at the example, the developer set descriptors, that is mapped internally in the browser

Add softmax and relevant element-wise binary and unary ops

Add softmax and relevant element-wise binary and unary ops (PR #68)

anssik: Softmax op required by the first-wave models
… this big op decomposes into smaller ops reduceSum, div and exp
… In addition to those smaller ops, this PR added alltogether the following:
… element-wise binary operations: sub, mul, div, max, min
… and element-wise unary operations: abs, ceil, cos, exp, floor, log, neg, sin, sqrt, tan, tanh

anssik: these binary and unary ops are reportedly supported by both ONNX and TensorFlow
… it seems there's one comment from Chai that Ningxin address today
… otherwise we seem to be good?

ningxin_hu: that was an accurate description of the status
… the last comment from Chai was a great one, by convention we added sample how to decompose this into primitives
… within the implementation I followed a practice that avoid numerical instability issue when dividing with a very big number
… Chai proposed we document this trick and reference
… fixed that with a new commit in the PR

Chai: LGTM
… we also do use this trick in DirectML
… adding a reference makes sense

anssik: comments?

Noise suppression

Evaluate noise suppression models for required ops (issue #66)

anssik: wanted to do status check on the next steps
… since last meeting, RNNoise author Jean-Marc Valin (@jmvalin on GH) chimed in on the issue offering a helping hand
… Chai proposed NSNet model as another evaluation target besides RNNoise

NSNet paper

anssik: Chai also noted both RNNoise and NSNet are hybrid DSP/ML based solutions, wondering how the DSP part integration Web Audio will work

Chai: looking at both the papers, there shouldn't be any problems with functionality re FFT windowing
… we need someone who understands Web Audio implementation in browsers to gauge feasibility, asked Jean-Mar about this on the issue

WebNN API Explainer

WebNN explainer

anssik: we had a chat with WebNN editors about position of explainer and the spec. In short explainer is an informative reference that helps folks get up to speed with the API fast and facilitates reviews, for example W3C TAG asks for an explainer at review-time
… our spec is currently light on introduction part so we may move informative text between the spec and explainer freely.
… examples are another important informative part of the spec and/or explainer, we have an issue for that

Add a more advanced example (issue #70)

Chai: is it the MNIST dataset?

ningxin_hu: Looked at LeNet-5 topology, also looked at ONNX.js example
… it uses a similar architecture

ningxin_hu: for the WebNN example purposes, we need to include the code to construct the graph
… if I work on this example, I'll rewrite it from the scratch
… we could discuss the idea of including the polyfill as part of the project

Ping: where are you planning to train the model?

ningxin_hu: for the example, I may use pretrained network from MNIST dataset

[possibly related to the discussion: https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌74]

ningxin_hu: idea is to be format independent, weight can be in any format
… or can load weights from a separate file, very flexible, not part of the example to explain that, the scope is to show how to use WebNN graph builder to utilize LeNet architecture

Ping: having the weights directly in the code would make for a clearer example

Chai: normally weights are packaged, but they layout needs to be supported by the model, for the purpose of the example, best to minimize deserialization and focus on model construction part of the sample, with weights inline it would be harder to understand the core functionality

ningxin_hu: need to agree this is a meaninful problem to solve as an example

anssik: proposal is to add an example of convolution neural network (CNN) for handwritten digit classification

<paul-mcdaniel-msft> note: that mnist model we are talking about has about 12 nodes. should be a nice and simple model to use as our first set . very small graph.

<paul-mcdaniel-msft> https://‌github.com/‌onnx/‌models/‌blob/‌master/‌vision/‌classification/‌mnist/‌model/‌mnist-8.onnx

Paul: seems like a great model to start with

<Chai> +1

<wonsuk> +1

anssik: any concerns to add the proposed example?

Ping: this is not a very typical use case
… but a good demonstration of the API
… a simpler would be better, but agree we can start with this

PROPOSED RESOLUTION: Add an example of convolution neural network (CNN) for handwritten digit classification

anssik: any concerns with this proposed resolution?

[none]

Resolution: Add an example of convolution neural network (CNN) for handwritten digit classification

AOB

anssik: our specs how have a shared visual identity with a logo :)

anssik: We may cancel some meeting in July/Aug to account for the vacation period, watch the mailing list for meeting info.

Adjourn

Summary of resolutions

  1. Add an example of convolution neural network (CNN) for handwritten digit classification
Minutes manually created (not a transcript), formatted by scribe.perl version 121 (Mon Jun 8 14:50:45 2020 UTC).

Diagnostics

Succeeded: s/do t/do use this trick in DirectML

Succeeded: s/... comments/anssik: comments/

Succeeded: s/us/use pretrained network from MNIST dataset

Succeeded: s/xample/example

Maybe present: anssik, Chai, Jonathan, Paul, Ping, Rafael