Meeting minutes
Repository: webmachinelearning/webnn
Announcements
Daylight Saving Time (DST) change
anssik: we're in the Daylight Saving Time (DST) transition period. This means in the US and Canada this meeting takes place today exceptionally at 8 AM. Next meeting will again be back at its normal US/Canada 7 AM.
… PRC will be an hour earlier at 22 PM for the next ~6 months starting with the next meeting.
Documenting implementation status
anssik: issue #223
<gb> CLOSED Issue 223 Documenting implementation status (by anssiko) [process]
Implementation Status of WebNN Operations
anssik: Thanks to sustained major contributions by Belem we now have:
… - a detailed implementation status page that covers both browser implementations and JS ML framework integration status for the WebNN API
… - upstreamed a schema-compliant subset of this data to the Browser Compat Data (BCD) repository used by MDN, caniuse.com, and more as a data source
… - published contributing guidelines to help new contributors take part in this effort
… Thank you all contributors! Details in the issue. Example caniuse.com entries:
Understanding and managing the impact of ML models on the Web
<dom> AI & the Web: Understanding and managing the impact of Machine Learning models on the Web
anssik: The W3C Team has documented its shared understanding of the current and expected systemic impact of AI and ML models in particular on the Web. We'll hear an introduction to this work that references and in part builds upon this group's efforts including WebNN API, Ethical Principles for Web Machine Learning, Hybrid AI exploration, and more.
Understanding and managing the impact of ML models on the Web
Comments welcome via GitHub issues
dom: thanks Anssi for inviting me to talk about this document
… I've tried to summarize many many conversations in this space, take systemic approach to this, not just one instantantion
… UA deployments may have broad long-lasting impact on the web as we know it, a classical example is AI generated content possibly polluting the web and making it harder to find quality content on the web
… something that has existed for decades, creating challenges in that space, with AI the scale changes and creates a more systemic risk to look at
… this document may be incomplete, also wrong in some cases, but wants to build a shared picture how AI is changing the web so we put in place the right mitigations and take the right actions to get the good parts of the AI and Web
… specific focus of the doc is on ML models and a lot of it on LLMs and generative AI broadly speaking
… as Anssi said, the doc refers to WebNN that offers some privacy mitigations for the web, builds upon the Ethical Principles work also deliverable of this WG, also noted Hybrid AI as a potential new way of handling privacy-preserving challenges
… how this WG could help, please review the doc, identify bugs and gaps, even better because you're well connected who care about AI in your orgs, please help make sure these people also take a look and bring feedback
… this will help build better shared understanding what this community needs to do and what W3C should drive forward
… feedback via GH or directly to Dom via email
anssik: will this doc be a Living Docoment?
dom: for now Living Document, living plan is to share this in the upcoming AC meeting in Hiroshima next month
… next big milestone around TPAC 2024 in California in September, expecting to organize a breakout session to say where we are, what has gained the most interest, come up with a roadmap for new work
… a lot of this depends on what contributions we get from the community
… we could also organize another W3C workshop, as you recall, this WG was created as a response to the first W3C Web & ML workshop
dom: we talked with Web & Network IG re where models are located, executed and how the data is handled
anssik: on a related note, thanks to everyone's feedback for the Hybrid AI proposal!
… the project team has posted a summary of your comments and wants to acknowledge everyone's contributions
… please check that the summary reflects your feedback:
webmachinelearning/
<gb> Issue 5 Hybrid AI Exploration (by grgustaf)
anssik: as the next step the team will share a proposal for the initial prototype whose scope is informed by your feedback
MLBuffer update
anssik: Update to the group on the latest developments in the MLBuffer exploration, learnings from the implementation. Spec design discussion to resume informed by the implementation findings.
Bryan: brief update, non-interop prototyping has been ongoing per original plan
… will triage the corner cases once the prototyping finishes, doing performance testing side by side
… on the spec side there's some work that can be done in preparation, e.g. timeline issue, that'd help us get prepared
… the prototype is expected to reach its first milestone in the next quarter
… for spec working on timeline and interop
Open issues and PRs
anssik: I see a lot of great progress in addressing open issues with many important PRs merged.
… let's first catch up with PRs merged since our last meeting, get a debrief from the authors as appropriate, and see what remains for us to be discussed today
Debrief on PRs merged recently
anssik: these are the substantive PRs that were merged in the last two weeks:
- Content: Define operand concept, simplify graph connection steps #591
- reshape(): Remove "nullable" in prose description of newShape #601
- Styles: simplify algorithm markup, improve step & dfn visibility #594
- Content: Define output for gru() #600
- Content: Clarify the expected behavior for axes being empty #599
<gb> MERGED Pull Request 600 Content: Define output for gru() (by inexorabletash)
- Content: Define build() steps more rigorously #603
- Synchronously validate input operands/validations #605
- Drop definitions for "platform operator" and "platform operand" #606
- Bugfix: Add missing input operands and activations #609
- Bugfix: Updated the step of calculating output sizes by scales for resample2d #611
<gb> MERGED Pull Request 605 Synchronously validate input operands/validations (by inexorabletash)
<gb> MERGED Pull Request 603 Content: Define build() steps more rigorously (by inexorabletash)
<gb> MERGED Pull Request 609 Bugfix: Add missing input operands and activations (by inexorabletash)
anssik: anything the authors of these PRs want to bring to the wider group's attention?
jsbell: thanks everyone for the reviews!
… build algorithm and programming model PR
… issue #567 is worth debriefing on this call
<gb> Issue 567 Can an MLGraphBuilder be reused? (by reillyeon) [question]
jsbell: Reilly asked if the current behaviour is desirable?
… If MLGraphBuilder can't be re-used, then input() can synchronously validate that input names are unique.
… If MLGraphBuilder can be re-used, then at build() time the graph needs to be traversed to identify inputs connected to the outputs, and uniqueness validated then.
… spec could be simplified if graph can't be reused
MikeM: thinking of use cases for reusing graphs, progressive model download
… updates in place would be convenient
<jsbell> Occasional glitch slips through, but the team catches it
<McCool> (comment: however, we could also rebuild graphs from scratch in case we need to do updates - so "freezing" a graph once it is compiled is not a blocker - but would be good to clarify behavior either way).
jsbell: in Chromium implementation inputs and constants are not allowed to be outputs? No-op graphs, are those allowed?
Define build() steps more rigorously
<gb> CLOSED Issue 552 Decide how to specify graph initialization (by a-sully)
<gb> CLOSED Issue 457 Complete the build algorithms (by zolkis)
<gb> CLOSED Issue 448 Add traversal algorithm to buildSync() (by zolkis) [bug]
anssik: PR #605 merged as noted, fixes all the above-mentioned issues
<gb> MERGED Pull Request 605 Synchronously validate input operands/validations (by inexorabletash)
anssik: everyone good with this resolution?
<dwayner> "No-op graphs, are those allowed?" I didn't have a strong opinion either way on that, but conceptually a nop graph seems useful, if you have a pipeline that stitch together and have optional parts, but also you can trivially insert an identity operator into the empty graph to satisfy it.
Synchronously validate input operands/activations
anssik: issue #572
<gb> Issue 572 Synchronously validate input operands/activations (by inexorabletash) [bug] [question]
anssik: merged PRs #591 and #605 addressed parts of this issue
<jsbell> Remaining work in webmachinelearning/
<gb> Issue 572 Synchronously validate input operands/activations (by inexorabletash) [bug] [question]
anssik: Josh identified the following as remaining work:
- Rewording methods that vend MLActivations
- Standard phrasing for "be an operator for ..."
- Introducing "Validate arguments" section for each method
- Introducing "Calculate output shape" section for each method (maybe "output descriptor" is better?)
jsbell: do we want to introduce structure in op creation? We've done that on an ad-hoc basis, validating inputs and calculating outputs are intertwined
… discussion on that issue welcome
Core operator set
anssik: issue #573
<gb> Issue 573 Core operator set (by philloooo) [question] [opset]
anssik: checking in to make sure we keep this issue on top of mind
… Josh you mentioned on our previous meeting your folks are working on an additional contribution
… would you like to discuss this today or should we defer this topic to a later meeting?
jsbell: we should defer, while important as additional implementations such as CoreML implementation is better fleshed out first
Consider adopting new broadcasting rules
anssik: issue #590
<gb> Issue 590 Consider adopting new broadcasting rules (by a-sully) [question]
anssik: a proposal from Austin to revisit broadcasting rules
… the rationale for the proposal, WebNN specifies unidirectional and bidirectional broadcasting rules, only ONNX makes this distinction, some others don't e.g. NumPy, PyTorch, Tensorflow/XLA/StableHLO
Dwayne: technically NumPy and XLA do implement unidir but don't give it a name, I think we can carry this on in the issue
anssik: Dwayne asked for more info on BNNS and MPS behavior, maybe Mike_Wyrzykowski can help?
MikeW: I take a look at that
anssik: thanks!
Is "validate graph resources" backwards?
anssik: issue #602
<gb> Issue 602 Is "validate graph resources" backwards? (by inexorabletash) [question]
anssik: Josh identified an issue in the validate graphs resources algorithms due to which compute() method does not fail if inputs are missing or unused
anssik: and offers two options as possible solutions:
… 1) Invert the test - iterate over the descriptors, and verify that there are matching resources
… 2) Make the test reflexive - the set of expected and passed resources must exactly match
jsbell: well summarized, wasn't sure if the intent is to pass those input and only those inputs
anssik: any other feedback?
Consider using label to allow better error handling for async errors
anssik: issue #585
<gb> Issue 585 Consider adding node `label`s for more diagnosable error messages for async errors. (by philloooo) [feature request]
anssik: based on discussion this issue should be labeled "prototype implementation experience wanted"
… any prototyping activity ongoing, anything to discuss for this?
Dwayne: not aware of any implementation effort
jsbell: Google folks are not looking at this right now
… we'll park this issue for the time being
Rename inputSize variables as inputRank in algorithms
anssik: issue #588
<gb> Issue 588 Rename inputSize variables as inputRank in algorithms (by inexorabletash) [conventions]
anssik: small thing but removes ambiguity which is always a great thing in a spec
… I suggest someone pushes out a PR for review to solicit comments
Dwayne: thank you Josh!
Consider alternate styling/linking for method argument definitions
anssik: issue #574
<gb> Issue 574 Consider alternate styling/linking for method argument definitions (by inexorabletash) [question] [editorial] [conventions]
anssik: this globally applied styling proposal welcomes feedback, three options illustrated in the issue
… these styling issues are a matter of taste, but please put your vote in!
… it looks like option 1 is currently winning
jsbell: I want to follow up with Bikeshed folks to make it easier for authors
… so not jumping on this PR immediately
Hybrid AI exploration update
<gb> Issue 5 Hybrid AI Exploration (by grgustaf)
MikeM: looked at all feedback, summarized that to build shared understanding
… we have some prototyping ideas, want to emphasize this is exploratory phase so happy to course-correct per feedback
… large models and download latency, cross-origin, built-in model reuse are big ticket issues
jsbell: thank you for your exploration here!
… you probably saw Jason/Google's feedback on this issue
… back in the day jQuery downloading worked with CDNs but no longer due to privacy improvements and especially for big models
… MikeM: we'll look at the issue tracker carefully and listen to your feedback
MikeM: doing requirements gathering now and that's why this group's input is so important
<anssik> s/… we'll/MikeM: we'll