Meeting minutes
Announcements
TPAC 2022 Hybrid meeting announcement
anssik: TPAC 2022 is a Hybrid Meeting
… takes place 12-16 September 2022
… Main in-person hub:
… Sheraton Vancouver Wall Centre
… Vancouver, Canada
… plan to also have a hub in China to provide an in-person gathering place to support a minimum number of sessions.
… Goals:
… - ensure the safety of our onsite attendees
… - valuable experience to both in-person and remote
… - optimizing for hybrid
… - group meetings and breakout sessions
… TPAC planners to share more details on practicalities, bookings, registration, health rules etc.
… Registration to open early July.
… Looking forward to seeing you virtually or in-person!
… questions?
WebML and WebNN logos
anssik: this WebML effort has been noticed outside this W3C group so we thought this effort deserves its own brand identity
… a core part of that identity is a logo that distinguish this effort in the minds of users, developers, other stakeholders
… Belem has created and contributed to this effort a set of beautiful WebML and WebNN logos licensed under CC BY 4.0 (Attribution 4.0 International).
… I look forward to t-shirts and coffee mugs with these logos
… I asked Belem to tell a story how these logos came to be and Belem joined us today to share his story with you
Belem: I'm excited to bring these new logos to the community
… abstract version of a neural network inspired me, I simplified it for this purpose to be more abstract
… this represents this emerging technology we're bringing to the web with you're help
… colors match W3C logo's colors
<ningxin_hu> thanks belem
<chai> looks great on dark background too!
Proposed new use cases
anssik: we drive API design by use cases
… we have two proposed use cases in review
Performance Adaptation
Performance Adaptation use case
… Performance Adaption use case has been discussed earlier, and it seems based on review comments the proposed use case is not practical
… any concerns with us deferring this use case from the CR scope and continue refine it for post-CR?
[no concerns]
Ethical Content Filtering
Ethical Content Filtering use case
anssik: Humera submitted this Ethical Content Filtering use case
Humera: the background for this use case is how the browsers can restrict ML implementations, and if this happens we're not able to perform certain things like content filtering important to users
… we thought of a few scenarios, e.g. user privacy
… content filtering should not be blocked by browsers, we think
… ML based content filtering is possible use case
anssik: current implementation is implemented as a browser extension?
James: thinking this through, "content filtering" vs "content filtering"
… how do we say filtering is ethical?
… people could use filtering to e.g. remove political opinions they do not like
… how do we bake that in, so this will only enable ethical and not non-ethical content filtering
Humera: type of content is a question a developer needs to consider when using this technology
Context-based graph execution methods for different threading models
PR: Context-based graph execution methods for different threading model
anssik: extremely important issues, huge thanks to Chai, Ningxin, Rafael, Bryan, Dom, Ping, everyone for your contribs
Chai: I closed the first PR #255, so we focus on #257
… essentially, to recap, we're now at the point we support both sync and async
… for sync per Ningxin's feedback we added GPU support
… we didn't do that initially due to concerns on GPU blocking UI thread
… but we agreed sync exec is only on worker thread we're OK
… for async it is clear, the use case is TF.js and Ping drove this use case, no controversy here for supporting both GPU and CPU
… we have covered the cases where the developer does not want to define the device explicitly
… "I want to use CPU"
… simple and straightforward
… in case of device selection, before this PR we had a hint, default too that leaves selection to the user agent
… so caller defers the decision to the UA, problem with implicit device type aka hint is that when you exec the model you'll have to choose what form of inputs or constants you want to feed the graph
… simpler to not having to think about that but there's a price to pay
… WebNN is a backend API for frameworks
… in almost all cases fw knows which device they want to use
… so a change is to have device type not being a hint but a choice
… making it explicit what type of processor the caller wants to hit
… harder case is when interopping with WebGPU, then the caller is mingled on the device level, need to choose adapter, map it etc.
… inefficient to defer to WebNN to create another device or a proxy
… for WebGPU the caller want to pass the WebGPUDevice and tell WebNN use what you have
… cannot be just a hint in this case, we want to make it clear there's an overload to create the context
… context has a dedicated method to create a new context out of WebGPUDevice
… caller using WebGPU, we have another overload that takes WebGPU context to create WebNN context, when we interop with WebGPU a fundamental issue is that the nature is to control how you populate the WebGPU workload and execute it
… WebNN in this model cannot exec the model on behalf of the caller, because it needs to maintain control how the workload is sequences and submitted for execution
… the additional step is to record the workload into WebGPUBuffer and submit it later
… recording the workload belongs to WebNN, best implemented synchronously because it happens on CPU and is a quick operation not blocking
… but the return is in a form of a workload that is "encoded"
… we press forward with our MLCommandEncoder proposal
… interops with GPUCommandBuffer, so WebGPU can consume it
… my preference would be to make it fully compatible with WebGPU instead of having another indirection
… that might fragment WebNN implementation from the caller point of view, using WebGPU already
… this was extensively discussed in PR #257
… from the API standpoints this seems the better choice of the two
… last commit done last night
… I hope we can agree on this and move on
ningxin_hu: great job to put this together!
… this satisfies a very complex and hard requirements from different sides
… I'd like to capture the usage perspective
… as Chai mentioned has two major usages
… 1) web developer using WebNN standalone
… we call this default ML context, allow selecting CPU or GPU, and is not a hint anymore, makes sense for framework integration
… for Wasm integration we need CPU device to avoid memory copies
… also use case to offload to GPU, we have the device type as an explicit instead of a hint will satisfy this use case, so the data is in the device that is requested
… we have the async API to avoid blocking and sync API restricted to worker to address concerns from TAG and also feedback from Chrome team
… default ML context for both sync and async accept ArrayBuffers
… UL/DL is implicit done on behalf of the caller
… I think this makes the WebNN default usage self contained
… 2) WebGPU interop, for this we have use case for real-time video processing
… this use case investigation needs a capability to combine both the rendering API WebGPU and WebNN graph exec capability together to implement e.g. background blur for video stream
… from my experience implementing this, developers start from constructing GPU pipeline, fitting ML capability into this pipeline, so WebNN should be addition to the existing pipeline
… CommandEncoder fits this model well
… WebGPU develop can submit the buffers to WebGPU Queue, Raphael shared new WebGPU developments to enable multiple queues
… in a summary, current WebGPU interop is in good shape and I'd like to support that
… there's a remaining ML context than can be created from WebGL rendering context, I believe we leave that for another future PR as it needs further investigation
… WebGL is a Khronos specification
… for this PR, we are close to meet the requirements, remaining discussion with Chai on graph initialization, but we can do that in a follow up PR
… this PR should be merged as soon as possible and work on additional enhancements in other PRs that come later
anssik: Chai, Ningxin any major blockers you'd like to seek WG perspective on?
Chai: I think we have a plan to get this PR to closure
Candidate Recommendation maturity wide review expectations
Accessibility review
anssik: This is a draft checklist to support Framework for Accessibility in the Specification of Technologies (FAST) prepared by the Accessible Platform Architectures Working Group.
… The goal of FAST is to describe the features that web technologies should provide to ensure it is possible to create content that is accessible to users with disabilities.
… WebNN API defines an API so the clearly relevant section is "If technology defines an API", checkpoints:
… [ ] If the API can be used for structured content, it provides features to represent all aspects of the content including hidden accessibility features.
… [ ] If the API relies on user agents to generate a user interface, the specification provides guidance about accessibility requirements needed to enable full interaction with the API.
… I can take the first stab at this checklist and submit a proposal for WG review
Internationalization review
Internationalization checklist
anssik: similarly to accessibility, given WebNN API is a low level API, many internationalization considerations do not apply and this review is expected to be a light weight
… happy to take care of this as well
WebGPU review
our plan of record from https://
… land PR #257 Context-based graph execution methods
… prototype that in Chromium
… update the WebNN-WebGPU interop samples accordingly
… then seek WebGPU review
… everyone happy with that plan still?
<ningxin_hu> +1 to this plan
anssik: thanks everyone for joining and for your active participation in GitHub! Please review the open PRs.