Happy Halloween! 🎃 It seems we lost a bunch of folks due to halloween celebrations. 👻
anssik: Review POC requirements for buffer sharing between dedicated ML hardware and GPU
… Ningxin has done investigation with Paul
ningxinhu: no slides to share, but can update on the status and next steps
… we need to investigate buffer sharing and performance
… we reported at F2F the results from WebGPU
… use case is to share WebGPU buffer, and run shader on GPU device and interact with ML accelerator
… we have Chromium WebNN POC implementing the foundation API
… this is based on native platform APIs, decided to use DirectML as a platform to do this investigation
… with DirectML we have WebGPU with D3D12 backend
jdarpinian: at least Dawn backend is confirmed
ningxinhu: WebGPU support with Chromium fork rebased to 79 so will check WebGPU support status
… in our fork WebNN works on top of DirectML, also supports Intel's VPUs on some devices
… used AI devkit, a small PC with VPU, PCIe interface for the investigation
… got help from Paul McDaniel of Msft
<ningxinhu> https://docs.microsoft.com/en-us/windows/win32/dxcore/dxcore-enum-adapters
ningxinhu: DirectML only in Windows insider builds, so testing with those preview releases, API enumerating accelerators only works on GPU devices, new interface dxcore in the insider builds only
ningxinhu: to summarize, we get GPU device to work on WebNN DirectML backend via DXCore, preliminary testing demonstrated it works
… VPU functionality confirmed, interop testing with WebGPU needs more work, is a next step
… can WebGPU compute shaders run on dedicated accelerators, VPU?
… feedback requested from community on the interop testing approches
anssik: can you open an issue to document the interop testing results to date and proposed next steps?
ningxinhu: can do that
jdarpinian: no plans to support VPUs via WebGPU I suspect, focused on GPU
ningxinhu: VPU just an example, WebGPU built on top of D3D12, so can WebGPU shaders run on compute-only devices?
jdarpinian: I think eventually yes, but not for the WebGPU v1, not 100% sure though
jdarpinian: any idea of relative perf of VPU and GPU in this machine tested on?
ningxinhu: we can run some models with GPU and VPU, GPU slightly faster than VPU currently, SqueezeNet <10 ms, VPU >10 ms
ningxinhu: the performance is one angle, VPU more power efficient
ningxinhu: jdarpinian mentioned WebGPU extensions a while ago, any updates on that?
jdarpinian: WebGPU CG is open to having ML extensions, but they're not going to develop that themselves, would need to be done in WebML CG
… I still think it would be a faster, simpler path, but less capable
ningxinhu: 4 opens for conv2d to be resolved
<jdarpinian> anssik, you are breaking up
<jdarpinian> it is very difficult for me to hear you
ningxinhu: not much bandwidth to help with native mapping for matMul
Web Neural Network API Explained
anssik: Explainer is a landing zone for early ideas, when formalized land in spec
… workflow: new issue -> discussion -> early consensus -> add to explainer and/or spec -> add to spec
… couple of open PRs waiting to be reviewed:
anssik: Fukuoka F2F had training capability agenda topic, but skipped due to lack of time
… wanted to discuss key use cases, feasibility, platform support (Core ML 3, ONNX 1.7 ...)
… currently training out of scope, so need group agreement to work on this
GabeEsteves: I don't have info on platform support, but I have use cases for API support
… use cases transfer learning and personalization area, customizing the last layers of the network
… I know e.g. Core ML and TF have had this capability since beginning
… ONNX 1.7 will also get this, expected to be released this month
… industry is moving to this direction, so perhaps the group should pay attention and track this area
The APIs in scope of this group will not be tied to any particular platform and will be implementable on top of existing major platform APIs, such as Android Neural Networks API, Windows DirectML, and macOS/iOS Metal Performance Shaders and Basic Neural Network Subroutines.
GabeEsteves: belem has more use cases in mind, specific concrete cases are recommendation engine personalization, speaker recognition, in order the web to be fully featured platform this capability would be useful
… can put this GH issue for further feedback and comments
anssik: propose the group focuses on inference near-term and keep an eye on training capability developments, related platform APIs etc.
gabe: agree with that positioning
Succeeded: s/this group/WebML CG/
Maybe present: anssik, gabe, GabeEsteves, jdarpinian, ningxinhu