W3C

– DRAFT –
WebML CG Teleconference – 27 June 2019

27 June 2019

Attendees

Present
Anssi_Kostiainen, Daniel_Smilkov, Ganesan_Ramalingam, Greg_Whitworth, Jonathan_Bingham, Kai_Ninomiya, Ningxin_Hu, Rafael_Cintron, Tomoyuki_Shimizu
Regrets
Chair
Anssi
Scribe
Anssi, anssik

Meeting minutes

Graph-building syntax simpler for web developers - review proposed API changes

<anssik> https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-502921949

<anssik> ningxin_hu: thanks Rama for comments and suggestions for improvements, based on Rama's key points

<anssik> ... two options: 1) proposal as in GH, 2) discussion with Rama

<anssik> ... Improvement areas: constants for operand type, no support of unknown dimension.

<anssik> ... Proposals: 1) rename OperandOptions with OperandDescriptor that actually describes a operand. 2) replace operand type constants with enum OperandType. 3) allow to use negative integers in the dimension list for an unknown dimension

<anssik> [Ningxin walks us through the details of the proposal documented in https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-502921949]

ningxin_hu: addOperand improvement areas:
… addOperand requires user code to maintain the operand index by
… an integer by itself.
… addOperand has multiple usages which would confuse developer.
… Proposals: 1. introduce Operand interface and use Operand object as return value.

2. replace addOperand with addInput and addConstant.

3. introduce Operand addInput(OperandDescriptor desc) for graph's inputs.

4. introduce Operand addConstant(OperandDescriptor desc, ArrayBufferView data) for constants.

5. remove the requirement of specifying op's output operand (see addOperation proposal).

6. replace usage of operand for op's attributes by dictionaries (e.g. ConvAttributes).

7. rename identifyInputsAndOutputs with identifyOutputs as inputs could be identified by addInput.

ningxin_hu: addOperation improvement areas:
… 1. addOperation requires to specify the shape of outputs that can be inferred.
… 2. addOperation requires to specify operation type by constant.
… 3. addOperation requires to provide op's attributes by adding operand.
… Proposals:
… 1. return Operand or sequence<Operand> when adding an operation
… 2. replace addOperation and operation type constants with add<OperationName> method for each operation
… 3. add Operand addAddition(Operand a, Operand b) and Operand addMultiply(Operand a, Operand b) as start and rewrite example with the new API

Rafael: do we have a PR with these changes?

ningxin_hu: we have a second proposal with Rama after some discussion, so some details might still change, thus no PR yet

ningxin_hu: second proposal at https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-504305802
… this second proposal decouples the operations from Model interface
… API design-wise, 1st proposal is TensorTR style while 2nd proposal more like nGraph and MPSGraph style

ningxin_hu: seeking feedback from the group on which proposal is preferred before crafting a spec PR

Rama: I have one concern: will it be clear which graph we're adding to if constructing multiple graphs e.g. if we have control flow

ningxin_hu: at this stage not linked to any graph, nn.createModel() specifies the output operand

Rama: that is a problem if users create multiple models from the same nodes, do you creates copies in that case?

Rafael: functionality of the API seems fine to me, modern Web APIs seem to prefer constructors over factories

anssik: old proposal https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-502921949

new proposal https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-504305802

dsmilkov: I prefer the second "new proposal", reason is it brings it closer to eager mode
… secondly, graph is implicit on how the nodes are tied together, which is good, since you can fork
… you can e.g. multiply one side, this would require re-adding ops in the first "old proposal"

Rama: minor comment, sharing should be possible in both the proposals

tomoyuki: no preference at this time on the two proposals

anssik: hearing slight preference for second aka "new proposal"

anssik: any objections to advance with the second proposal?

Rafael: fine with this, but suggest we try adopt constructor pattern
… also nn.constant not clear you're adding something to the graph

[hearing no further comments or concerns]

PROPOSED RESOLUTION: Craft a spec PR based on https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-504305802 considering feedback on the call

Resolved: Craft a spec PR based on https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-504305802 considering feedback on the call

Survey graph-building APIs from native ecosystem - discuss TensorRT findings, survey conclusions

https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌20#issuecomment-501255805

ningxin_hu: this survey informed the discussion in issue #16

[summarizing how each of the native API survey helped us understand on how to design a Web API that integrates with frameworks]

anssik: any further input for #20?

ningxin_hu: when #16 is fixed, close issue #20

Define the set of operations and their specification

gregwhitworth: bringing back this discussion we had a month ago, jonathan & co wanted to get back to this
… would like to understand where we are with this issue

Jonathan_: followed up on the proposal re ONNX internally at Google, our involvement with ONNX has been minimal, not because opposition, we've just been busy -- still need to find a person from Google side to review this in more detail
… some open questions, e.g. should this be a layered API?

gregwhitworth: I'm curious what the next action would be, not disagreeing exploring other operations beside those defined in ONNX meanwhile the issue is being discussed

<gregwhitworth> https://‌github.com/‌onnx/‌onnx/‌blob/‌master/‌docs/‌Operators.md

kainino: asking for a reference what ONNX defines
https://‌github.com/‌onnx/‌onnx/‌blob/‌master/‌docs/‌Operators.md seems not very precise

gregwhitworth: if more precision needed, then that should be discussed in ONNX project

kainino: some of the aspects are more precise than other areas

gregwhitworth: agree with Jonathan it is good to consider other holistic ideas but that should not block us

<gregwhitworth> PROPOSED RESOLUTION: The specification will reference the ONNX operations and if there are any improvements desired for ONNX the work should be there.

dsmilkov: we want to make further investigation into the ONNX operations being referenced

https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌17

<gregwhitworth> Per the proposed resolution, we'll allow two weeks for folks to review and then resolve on the next telecon

[skipping Custom operations - framework integration case study report out: WebNN backend prototype for ONNX.js in the interest of time]

Any Other Business

holiday season is on, next call 8 August 2019

Adjourn

Summary of resolutions

  1. Craft a spec PR based on https://‌github.com/‌webmachinelearning/‌webnn/‌issues/‌16#issuecomment-504305802 considering feedback on the call
Minutes manually created (not a transcript), formatted by Bert Bos's scribe.perl version Mon Apr 15 13:11:59 2019 UTC, a reimplementation of David Booth's scribe.perl. See history.

Diagnostics

Succeeded: s/API design-wise, 1st proposal is TensorTR style while 2nd proposal more like nGraph and MPSGraph style/... API design-wise, 1st proposal is TensorTR style while 2nd proposal more like nGraph and MPSGraph style

Maybe present: anssik, dsmilkov, gregwhitworth, Jonathan_, kainino, Rafael, Rama, tomoyuki