– DRAFT –
WebML CG Teleconference – 14 March 2019

anssik: We have three topics on the agenda: executing operations, custom operations, industry events
… let's spend as much we need for the first two, time allowing discuss the last one

Executing operations

Executing operations"

anssik: On our path to API spec we should now derive requirements that will inform the design of the API for executing operations.
… since our last monthly sync:
… Ningxin provided a summary of platform APIs and their supported executing modes

anssik: Nikhil proposed that for platform APIs (e.g. NNAPI) that need a graph we use an implicit graph that allow decorating functions in order to convert them to subgraphs.

anssik: Rafael noted the API should enable the use of specialized ML hardware.

anssik: Let's discuss. The expected outcome of this issue would be a list of requirements we can them feed into the spec to use as a checklist as we advance to the API definition phase

[ revisiting discussion in the issue ]

<Daniel_Smilkov> If TF.js, operations like neg() and square() would return a logical wrapper around the resulting data, but not the data. When user calls result.data(), we transfer the data back to user-code

<paulmcdaniel> this is paul, i would like to propose we capture this proposal with some more details in the github issue and also iterate there. i'm not sure this will work or not for our hardware acceleration

<paulmcdaniel> it might work great, but i have questions and can also show what we require via pictures/IDL in the github iteration

Custom operations

-> Custom operations

<ningxinhu> https://‌docs.google.com/‌presentation/‌d/‌1xOQtyAeQNdI70rICzLNKRxmzM-Dwjhh3MdzGKW29C8Q/‌edit?usp=sharing

anssik: Ningxin volunteered to investigate custom operations support on CPU and will give an interim update specifically for the Wasm backend

WebNN Investigation: Wasm Custom Ops (presentation slides)

[ Ningxin presents the slides ]

anssik: The slides will give a recap on custom operations, WebNN POC API, perf data, summarize key findings.

Ningxin_Hu: Background: in ML field, the ops are evolving quickly
… Problem: big framework’s ops set (e.g. TF.js has ~200) VS. small WebNN’s ops set (focusing on hardware acceleration)
… Status: frameworks ops are implemented in WebGL and WASM, later will add WebGPU
… Solutions under investigation:
… - Framework partitions the NN graph based on WebNN’s ops set
… - Framework offloads supported sub-graphs to WebNN execution
… - Framework executes other sub-graphs by own ops
… - Require a high-performance data exchange between framework ops and WebNN
… - Device affinity may matter: WASM ops interop with WebNN/CPU, WebGL/WebGPU ops interop with WebNN/GPU
… - Start with WASM ops

[ Using WebNN POC slide walkthrough ]

[ WASM ops and WebNN graph execution ]

[ Workload: MobileNet V1 ]

[ Graph partition configurations ]

<paulmcdaniel> question on the IRC: these slides are great ! thank you ningxin ! is the POC source code link somewhere? would love to see how some of the ops where connected to BNNS.

[ Results: performance ]

[ Results: efficiency ]

Ningin_Hu: Summary:
… Offloading key WASM ops to WebNN can get good speedup
… Interop between WASM ops and WebNN graph has overhead
… Need to explore the browser-side optimizations, e.g. by

<gregwhitworth> I have to go catch my shuttle. I'll follow up on GH.

Ningin_Hu: Reduce memory copies: CPU backend inside WASM process? Shared WASM heap?
… Reduce memory relayout: expose native memory layout?
… Need to explore the framework-side optimizations, e.g. by implementing overhead-aware graph partition and delegate algorithm

anssik: thank you Ningxin, there's a question from paul

ningxinhu: the source will be published soon

<paulmcdaniel> thanks !

Industry events

[ skipping in the interest of time ]

Adjourn

anssik: thank you for your active participation.
… Next call is scheduled 11 April 2019, so we're back to the original time, i.e. US/Canada an hour earlier than today.

<Daniel_Smilkov> Have a nice day!

s//‌github.com/‌webmachinelearning/‌webnn/‌issues/‌11/Executing operations//github.com/webmachinelearning/webnn/issues/11/Executing operations

s//‌github.com/‌webmachinelearning/‌webnn/‌issues/‌11///