Meeting minutes
Intros
Anssi: welcome to the 2nd call of the Web Machine Learning WG
… a few new names - intros?
SungpilShin_ETRI: I'm joining to take part to the work on Machine Learning
… I work with Wonsuk
… we're interested about unew se cases for the Web Machine Learning
… that I would like to see discussed
Add a new application use case #182
Sungpil: we plan to contribute to the APIs of this WG
Anssi: welcome to the group!
WebNN API - First Public Working Draft
Anssi: WebNN was published as First Public Working Draft on June 22nd
W3C First Public Working Draft, 22 June 2021
Anssi: congrats to the group on this milestone!
anssik: "The first Working Draft of a technical report is called the First Public Working Draft (FPWD), and has patent implications as defined in the W3C Patent Policy [PATENT-POLICY]."
Anssi: our first step towards Recommendation
Call for Exclusions: Web Neural Network API
Anssi: in terms of the RF licensing, you received a call for exclusions email
… all Advisory Committee representatives were copied - they are the ones in charge of that process
Anssi: Google provided feedback in the Call for Consensus to move forward with FPWD publication
Google's input on WebNN CfC to FPWD
Anssi: recommendation to pay attention to rapid domain evolutions
… pay attention to developer demands
… take into account lessons from real-world benchmarks
… and pay attention to advances in complementary areas such as Web Assembly
… thanks a lot for that feedback, Jonathan - great input to building a consensus view of our work ahead
Jonathan: good summary - these aren't new concerns, we've mentioned these points before
… we recognize that there is value in providing a graph-based API despite other advances we've seen, given the time it takes to deploy a Web standard and given the expected benefits of an improved primitive
… multiple people have skepticism this is the long-term optimal approach, but we have good hopes that it can be at least a good stepping stone, if not the ultimate solution
<RafaelCintron> +1
RafaelCintron: will Google invest in in NN API as a graph API given the possible mitigations you mentioned?
Jonathan: one of the risks we had identified is the growth of operation set
… due to operations not being composable based on existing operations
… tensorflow is not seeing any slow-down in that growth at that moment
… the mitigation we're exploring in Google is finding the right set of lower level operations that can be used to compose other operations
… we're hopeful this can be achieved
… the NN API group is still working on the next iteration of that API
… a complete revamp of the existing API
… I'm not sure why they didn't complete the existing NN API with lower level APIs - will share more when I know more
Anssi: NN API is the Android NN API here
RafaelCintron: will it be a graph API or is still TBD?
<Zakim> wonsuk_, you wanted to discuss plan for implementation on Chromium
wonsuk_: are there plans to implement WebNN in Chromium?
Jonathan: right now the focus in Chrome is on the Model Loader API - separate from the work on NN in Android
… eventually, there may be different implementations across PC vs android
Anssi: implementations is also part of the next agenda item
W3C Recommendation Track flow diagram
Next steps
Anssi: the first step is what we've just done, FPWD
… next step is publication of 0 or more revised WDs
Automating publication of future WebNN Working Drafts
Anssi: we're set up with an automated publication system which allows us to publish WDs on a regular basis
… each merged Pull request will trigger a new WD publication
… WD are development versions of the spec, they don't necessarily represent consensus
… the following step is Candidate Recommendation (CR)
… reaching that stage requires fulfilling the technical requirements
… it's a signal to get a final review of the spec and a signal the spec should be implemented
… nowadays, in practice, implementations often start well ahead of CR
… and we expect to get wide review from the community before we reach CR
… CR is the time when this gets assessed
dom: horizontal group review expected latest at CR
… another aspect is testing the API, part of what is assessed during CR is how much implementation we're getting, web-platform-tests is the project that hosts cross-browser tests
… tests help implementers implement and assess implementation gaps
Anssi: the WebNN polyfill has a test suite - that might help down the line. Ningxin, where does that stand?
Ningxin: the webnn polyfill has a tests folder with mocha tests
Ningxin: we have an open issue to turn these tests into WPT tests
Ningxin: the porting of the test framework is still TBD, but the test cases themselves should be in a good shape
… the tests exercise at @@@ and operators
… there was discussion in the workshop around testing, with chai giving very good advice
… we have tests both at the operator and model levels
Conformance testing of ML APIs for the Web (from W3C workshop)
Ningxin: we have ported some of the native tests e.g. from Android NN
… and ONNX
… includes different memory layout to increase our coverage
… it's an ongoing effort
… we have a dozen tests for ops - the coverage is far from being complete
… contributions are highly welcomed!
Anssi: are we expected to use the WPT test harness?
dom: in practice it really helps to get browsers run these tests in their CI
Anssi: so translating the tests sounds like an important thing to do - is that something you can take on Ningxin?
Ningxin: will do
Anssi: in our charter, our timeline for reaching CR would be Q2 2022 - naturally, this is subject to our progress, based on consensus
… After CR, next step is Proposed Recommendation, the penultimate stage before the final status of Recommendation
… it triggers formal review by the W3C Advisory Committee, the whole W3C Membership
Dom: the real challenges are in getting into CR and out of CR - the rest is usually more of a formality
Supporting implementation efforts: webnn-native and webnn-polyfill
Anssi: Ningxin, can you talk about webnn-native and -polyfill, software projects from our sister Community Group?
… they inform the work on the WebNN API
… I expect we'll discuss the progress of these projects on these calls from time to time
… useful to highlight where contributions are welcomed in particular
WebNN Polyfill
Ningxin: the polyfill was started first - it gives a JS implementation of the API, based on tensorflow.js - thanks to Ping and his team for their help
… polyfills help with early implementation and usage experience
… as we discussed, the test suite was built in the context of the development of this polyfill
Samples that run with webnn-polyfill
Ningxin: In terms of contribution areas, the major thing is to keep alignment with API changes
Test runner for webnn-polyfill
Ningxin: e.g. recent device selection, pre-allocated output buffers, new operations
… we want to keep the polyfill as close as possible to the spec definition
Ningxin: this is reflected in the open issues of the repo
https://
Ningxin: we also have sample codes in the spec & in the explainer - they can be run through the polyfill which are made available through the webnn-samples repository
Ningxin: good way to explore new use cases for the API
Anssi: the polyfill is a good way to explore the API, easy to integrate into one's application
… very useful project
… many of the samples perform quite nicely with the polyfill
webnn-native
Ningxin: we started webnn-native with support from the Community Group
… the initial intention is to inform the CG (and now the WG as well) for the WebNN API design
… also helps with performance investigation
… it also provides C & C++ headers for applications to use, allowing WebAssembly use
… can be useful in transpiled projects like TensorFlow lite & ONNX
Ningxin: the repo has different backend implementations - DirectML, OpenVino
… it allowed running benchmarks with recent changes to validate them
Ningxin: In terms of opportunities for contributions, looking at open issues, it's again keeping the implementation close to the spec
… right now, it's a bit behind than the polyfill - implementation is a bit more involved than a JS implementation
Node.js binding to webnn-native
Ningxin: enabling running the tests would help determine conformance across different backends
… there are also backend-specific tasks - thanks to Chai and his team for their help on the DirectML backend
… one possible improvement would be to remove duplication with DirectML by upstreaming some of the code there
Backend implementations that use platforms' ML APIs:
DirectML on Windows 10
OpenVINO on Windows 10 and Linux
oneDNN on Windows 10 and Linux
XNNPACK on Windows 10 and Linux
Other backends are to be added
Ningxin: additional backends for other OS native APIs would be highly welcome
… combined with the tests, this would help detect good cross-platform support for operators
Anssi: both projects keep track of the spec changes closely, as reflected in issues
… webnn-native could be used as a backend to a chromium implementation
Wonsuk: amazing work, with both projects!
… thanks for your efforts!
… I'll try to get started with running the tests
WebNN API - TAG review
Anssi: WebNN has received review from the W3C Technical Architecture Group
[tag-tracker] Define a common term for logical tensor changes?
Anssi: most of the issues have been addressed, a few are still being worked on
Define a common term for logical tensor changes? #150
Anssi: Rama, you had a proposal to address this - would be good to turn it into a pull request, a good first contribution
[tag-tracker] Ergonomics of the JS examples
Ergonomics of the JS examples #139
… "RESOLUTION: Explain in the spec intro the rationale why the primary API consumer is a JS framework, note Model Loader API as a higher-level abtraction targeting web developers"
<ningxin_hu> sgtm
Anssi: the main story here is that the main users of the spec are expected to be framework developers, with a higher level API exposed to regular developers, e.g. model loader
[tag-tracker] String enum for activations
String enum for activations #138
anssik: no new information, Chai feel free to submit a PR
chai: working on this as part of the discussion on activation functions
anssik: great! finalizing our responses to these issues would be good
Help wanted!
Anssi: there are issues in our tracker that are welcoming comments and proposed solutions
… please review the list for triage purpose - I did some initial triage, classifying them into 5 buckets: proposed new features, to be clarified, ready to close, investigation and explainer updates
Anssi: please take a look and provide feedback
… if you have a solution in mind, please provide a pull request - the editors can help iterate
<chai> im working on a PR for #138, 181, 184 and 185
Anssi: there are a bunch of new ops being proposed for which we would need to look at decomposition, platform support, …
wonsuk: sungpil made a PR for a new use case for WebNN - given the time, we won't be able to discuss it today, so please look at the PR and give your opinions
https://
Anssi: I'll take a look
Summer break
anssik: These meetings will break for July and will resume mid-August
… Please respond to the meeting scheduling poll to signal your preferences:
WebML WG Teleconference scheduling poll
anssik: Note: the request is for a recurrent meetings, not the specific week (frequency to be confirmed, likely every other week for the time being)
… The GitHub repos will remain open for contributions while our meetings are on pause. Thank you for your continued contributions!
… Those of you who take time off: have a happy vacation, stay safe and enjoy!
<chai> Thanks Anssi. Enjoy the summer everyone!