– DRAFT –
WebML WG Teleconference – 15 December 2022

ghurlbot, this is webmachinelearning/webnn

<ghurlbot> anssik, OK

WebNN API Candidate Recommendation open issues

Current CR issues

Web platform tests

anssik: WPT tests tracker issue is #265

<ghurlbot> Issue 265 WPT tests tracker (BruceDai) cr

anssik: thanks Bruce for keeping the first comment in that issue updated.
… Bruce reports in that issue:
… - All submitted WPT WebNN tests PRs have been approved by Dwayne.
… - 414 WPT WebNN operations tests covered 39 ops of total 40 ops for first wave models.
… - Bruce is still working on last conv2dTranspose op tests.

Web platform tests: WebNN API IDL Tests

ghurlbot, this is web-platform-tests/wpt
… let's first discuss WebNN API IDL Tests

<ghurlbot> anssik, OK

anssik: first, IIUC WPT PR #37227 is the major blocker.

<ghurlbot> Pull Request 37227 Fixed CI failure of idlharness.js for unexposed overloaded functions. (BruceDai) infra, idlharness.js

anssik: PR #37227 fixes a CI failure introduced by the automated PR created by @webref/idl bot in PR #36908

<ghurlbot> @webref

<ghurlbot> Pull Request 36908 Sync interfaces/ with @webref/idl 3.22.3 (github-actions[bot]) interfaces

anssik: PR #37227 unblocks all the PRs in “1. WebNN API IDL Tests” category enumerated at https://github.com/webmachinelearning/webnn/issues/265#issue-1226221213 as well as all the Chromium CLs e.g. https://chromium-review.googlesource.com/c/chromium/src/+/4005570 that run wpt tests in Chromium CI.
… to ensure we meet the CR readiness criteria for testing
… I've asked Dom to help with WPT PR #37227 that lacks final touch review to unblock all WebNN API IDL Tests PRs and Chromium implementation CLs (blocked by Chromium CI that imports and runs wpt tests)

Bruce: #37227 also blocks WebNN op tests, I also asked Belem for help

Web platform tests: WebNN API JS Tests (testharness.js) for ops

ghurlbot, this is web-platform-tests/wpt

<ghurlbot> anssik, OK

anssik: thanks to Bruce for the heroic effort in creating these PRs and for Dwayne equally for carefully reviewing these PRs!

anssik: the following PRs are approved by Dwayne but at least 1 approving review is required by wpt repo reviewers with write access:

#37420 batchnorm

#36782 concat

#37067 clamp

#37275 conv2d

#37337 binary

<ghurlbot> Pull Request 36782 [webnn] Add tests for concat operation. (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37420 Add webnn batchnorm tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37067 Add webnn clamp tests (BruceDai) webnn, wg-s_webmachinelearning

#37351 unary

#37068 gemm

#37340 leakyRelu

<ghurlbot> Pull Request 37275 Add webnn conv2d tests (BruceDai) webnn, wg-s_webmachinelearning

#37379 pooling

#37069 matmul

#37380 reduce

#37070 reshape

#37071 relu

#37343 sigmoid

#37072 slice

#37254 softmax

<ghurlbot> Pull Request 37337 Add webnn binary tests (BruceDai) webnn, wg-s_webmachinelearning

#37398 split

#37352 squeeze

#37344 tanh

#37315 transpose

<ghurlbot> Pull Request 37340 Add webnn leakyRelu tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37068 Add webnn gemm tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37351 Add webnn unary tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37069 Add webnn matmul tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37379 Add webnn pooling tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37071 Add webnn relu tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37070 Add webnn reshape tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37380 Add webnn reduce tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37343 Add webnn sigmoid tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37254 Add webnn softmax tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37072 Add webnn slice tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37398 Add webnn split tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37352 Add webnn squeeze tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37344 Add webnn tanh tests (BruceDai) webnn, wg-s_webmachinelearning

<ghurlbot> Pull Request 37315 Add webnn transpose tests (BruceDai) webnn, wg-s_webmachinelearning

anssik: all these PRs have been reviewed by conformance testing expert and what is lacking is the sign-off from wpt repo reviewers
… I can ask Dom again for help to mass-merge all of these.
… to clarify, all of these PRs only touch files in the https://github.com/web-platform-tests/wpt/tree/master/webnn dir and its subdirs but not outside of it?

Bruce: yes
… thanks again Bruce and Dwayne!

Web platform tests: open webnn-baseline PRs

ghurlbot, this is webmachinelearning/webnn-baseline

<ghurlbot> anssik, OK

anssik: webnn-baseline is the pure JS implementation of WebNN API we use for testing purposes. Let's discuss and review the open webnn-baseline PRs first.

Open webnn-baseline PRs

#22

<ghurlbot> Pull Request 22 [tool] Add scripts to generate test data for WPT WebNN operations tests (BruceDai)

anssik: PR #22 adds Node.js scripts `gen-<operation-name>.js` in the `tool` folder to generate test data for WebNN operations tests of web-platform-tests
… approved by Dwayne, ready to merge?

Bruce: I will refine this PR according to the latest test data json file, I'm working on it and will ping folks for review after finished.

<ningxin_hu> sgtm

anssik: In addition we have the following open PRs that generate test data for various tests, all approved by Dwayne with some comments

#23 matmul

#24 reshape

#25 relu

#26 slice

<ghurlbot> Pull Request 23 generate test data for WebNN matmul tests of WPT (BruceDai)

<ghurlbot> Pull Request 24 generate test data for WebNN reshape tests of WPT (BruceDai)

<ghurlbot> Pull Request 25 generate test data for WebNN relu tests of WPT (BruceDai)

<ghurlbot> Pull Request 26 generate test data for WebNN slice tests of WPT (BruceDai)

anssik: are we ready to merge also these?

Bruce: I will refine the scripts in these PRs too similarly

Chai: Dwayne in on vaca until January, baseline feedback should be clear and easy to address
… I reviewed Dwayne's feedback and they are not suggesting major changes, so Bruce can address them easily
… once addressed we can merge, if approval needed I can do that
… Huge thanks to our key test contributors Bruce and Dwayne, Ningxin for making the WebNN testing story more complete and compelling!
… the improved test coverage provided by these PRs allow us to demonstrate implementation experience, an important CR readiness requirement

Use modern WebIDL conventions

ghurlbot, this is webmachinelearning/webnn

<ghurlbot> anssik, OK

anssik: another important area for CR readiness is to align the spec with modern WebIDL conventions and patterns
… Thanks to Zoltan for spearheading this editorial refinement effort!
… I've helped Zoltan by reviewing some of his PRs to ease the workload from our busy spec editors who are also delivering to other related commitments such as the Chromium implementation
… I've maintained the living task list of editorial work broken down into: Done / WIP / TODO

Living task list

anssik: The following editorial updates have been Done:
… - #309 Add MLGraphBuilder constructor steps

<ghurlbot> Pull Request 309 [closed] Add constructor steps to MLGraphBuilder (zolkis)

anssik: - #310 Improve createContext method steps and add method steps for createContextSync

<ghurlbot> Pull Request 310 [closed] MlContext improvements (zolkis)

anssik: - #311 Improve spec prose on "exists"

<ghurlbot> Issue 311 [closed] Improve spec prose on "exists" (zolkis)

anssik: - #314 Add method steps for createContextSync

<ghurlbot> Issue 314 [closed] Handle errors according to Web IDL conventions (zolkis)

anssik: WIP tasks are:
… - #308 Validate MLContext

<ghurlbot> Issue 308 Validate MLContext (zolkis)

anssik: - #316 Review sync vs async compute differences

<ghurlbot> Issue 316 Review sync vs async compute differences (zolkis)

anssik: Zoltan wanted to ask some questions re sync vs async compute differences to progress on this task

Zoltan: in #316 I noticed the following opportunities/differences:
… Inputs and their validation is the same in both, so it can be factored out and referred from both (done).
… The async steps currently are not really asynchronous. Also, they throw from a parallel job, so I assume that needs (minor) fixing (done).
… There is only one difference between the sync and async steps (excluding promise-related): the async version compares the byte length of |outputTensor| with the length of the output descriptor corresponding to |key|, whereas the sync version compares the same with the byte length of |value|. If that is deliberate, we need an editor's note on why so - please point me to relevant background there.

ningxin_hu: there are two pointers, refer to the previous WebNN implementation hosted on GH, that one is somehow outdated and comment in there is invalid
… I'll share links to the up to date implementation

Zoltan: interested in it sync or async steps we should follow, or should there be move checks? It looks like XNNPACK has more steps with checks

Ningxin: we should use XNNPACK inspired steps
… implementation experience can inform the algorithm steps
… some learnings there we can backport into the spec

Zoltan: I'm wondering if I'm looking at the latest code?

ningxin_hu: you're referring to the big CL for the Mojo implementation, that is not fully reviewed
… I suggest we align on the XNNPACK backend at its current state because it is being merged into the Chromium code and split into smaller CLs and merged
… I'll share mode details in the issue #316

Zoltan: execute and compute methods are on the graph, correct?

ningxin_hu: correct, IDL change Anssi mentioned was blocked by wpt infra PR and we are waiting for that PR to be merged for Chromium CI to fetch those tests, and then I plan to align the Chromium impl with the latest IDL
… impl itself is waiting for XNNPACK, JS binding can be bound to that, we can focus on the MLGraph impl

Zoltan: intention is to move compute to context in the C++ code too

ningxin_hu: correct

Zoltan: current spec refers to impl as an internal slot
… when we refer to impl it always means impl so we don't need an internal slot for that, but if there's another use for that internal slot we can use it

ningxin_hu: re internal slots, if you look at the Chromium impl, there's a dep in Blink on native lib to implement the native implementations
… e.g. XNNPACK depends on XNNPACK lib to execute the ops
… similar to DML backend in the big CL, there DML API will run the graph
… XNNPACK or DML backend is represented by the implementation internal slot
… maybe we can find a better way to define that
… internal slot represents the native library or OS API for ML component

Zoltan: this is common for many Web APIs, but they do not have internal slot representing the implementation

ningxin_hu: fine to use a convention that is widely used

Zoltan: will establish a convention and put up a PR for that, work with Anssi
… the rest of the algorithms mentioned in Anssi's tracking issue, Domenic's feedback was to document how inputs and outputs are handled towards the algorithms
… we don't need to specify how e.g. "bubble sort" works to give an example
… so expect short prose for those

Anssi: ... TODO tasks are:
… - Establish an algorithmic method steps convention for operations and apply that convention across all the ops.
… Thanks Zoltan for your editorial work that helps align the WebNN API spec with modern WebIDL conventions
… this work further increases our confidence we're reaching CR readiness

Zoltan: similarly tests should be aligned, so affects testing precision
… MLGraph is an opaque object in script, what are the plans there, are we going to add methods to it?
… I had some suggestions, but can discuss that later

Anssi: is MLGraph issue self-contained?

Zoltan: not yet clear, not clear for developers what it is
… we have builder spawn from the context

MLGraph is an intermediary step that could we an internal slot
… need to understand if there's a reason to keep it around

<ningxin_hu> https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/ml/webnn/ml_graph.idl

ningxin_hu: if you look at the Chromium code MLGraph is an opaque object so far, aligns with the current spec

.(pastes related code link)

Zoltan: in JS we don't usually have opaque interfaces, we hide it

ningxin_hu: MLGraph object represent a compiler computational graph, we should make this clear, e.g. in XNNPACK backend we inherit this interface

Zoltan: any methods we could expose on MLGraph such as shapes or dimensions?

ningxin_hu: I think that is a good idea, we could expose some methods to allow the developer to query input and output dimensions
… currently not proposed for the WG for discussion

Anssi: let's not rush this new feature, open a new issue to discuss this in the WG
… add an informative note that the opaque interface is there for future extensibility.

Zoltan: sounds good
… another concern that we have WebGPU and user given context
… why not have a single context that can be WebGPU, GPU, CPU
… I did some experiments and a good abstraction could be to appear on the context level, have CPU context, GPU context, WebGPU context, AI accelerator context

<zkis> https://github.com/webmachinelearning/webnn/issues/302

Anssi: I consider #302 v2 feature

<ghurlbot> Issue 302 API simplification: context types, context options, createContext() (zolkis)

Open PRs

ghurlbot, this is webmachinelearning/webnn

<ghurlbot> anssik, OK

anssik: we have two minor open PRs that emerged from Chromium implementation experience:
… #294 Use unsigned long for size related options of conv2d, convTranspose2d and pooling operations

<ghurlbot> Pull Request 294 Use unsigned long for size related options of conv2d, convTranspose2d and pooling operations (huningxin)

anssik: #306 Use unsigned long type for sizes option of resample2d

<ghurlbot> Pull Request 306 Use unsigned long type for sizes option of resample2d (huningxin)

anssik: these PRs are pending Chai's review, can you PTAL?

Chai: I'll take a look

Anssi: Thanks!

WebML WG Charter 2023-2025 proposal ready for WG review

ghurlbot, this is w3c/machine-learning-charter

<ghurlbot> anssik, OK

anssik: I'm pleased to announce the Web Machine Learning Working Group Charter for 2023-2025 is now ready for WG review.
… I'll now ask you to please review the draft PR and open issues, provide your comments and open new issues as appropriate to help shape the WG's technical scope.

Open charter issues

Diff to the initial charter

anssik: please see the Diff to the initial charter for changes
… this is the summary of changes I proposed:
… - Clarify in Out of Scope that "base linear algebra operations are out of scope" refers to the

Basic Linear Algebra Subprograms (BLAS) interface. (related #20)
… - Note WebGPU interoperability expectation in Coordination (fix #20)
… - Clarify WebRTC coordination (fix #21)
… - Add high-level to low-level op decompose path to facilitate progressive lowering by implementations (fix #22, related #28)
… - Expand Motivation and Background and clarify priorities in Scope (fix #23)
… - Add OpenXLA Project to Coordination (fix #28)
… notably, based on my assessment, these changes do not alter the Scope of Work and do not add any new deliverables
… to that end, this will minimize the need for internal legal reviews and my expectation is the current participants are not required to rejoin the WG when the new charter is operational and the transition is frictionless
… any questions or comments for any of the changes or for rechartering in general?
… Please provide your feedback by the end of the year. We are expected to start the AC review during Q1 to have an approval in time for a smooth transition.

<ghurlbot> Issue 20 WebGPU interoperability (anssiko)

<ghurlbot> Issue 21 WebRTC coordination (anssiko)

<ghurlbot> Issue 22 Level of abstraction for neural net operations (anssiko)

<ghurlbot> Issue 28 OpenXLA coordination (anssiko)

<ghurlbot> Issue 23 Set of ops supported must be more comprehensive (anssiko)

Happy Holidays!

anssik: Thank you all for a hugely productive 2022!
… Looking back, we've accomplished a lot this year.
… We've raced to bring the WebNN API spec to CR readiness quality for an early 2023 publication.
… We've made substantial progress in implementation on multiple backends across OSes.
… We have showcase WebNN-powered user experiences through running samples, and evolved the WebNN Polyfill and WPT tests and more!
… As we now start to transition to the holiday season, I want to wish those whom it concerns, relaxing holiday.
… please spend time with your family and friends.
… We'll skip the next WG call that overlaps with the holiday season and will be back re-energized 12 January 2023!
… See you in 2023!