Meeting minutes
WebNN API open PRs and issues
Repository: webmachinelearning/webnn
anssik: I'd like us to review the open PRs discuss key open issues
Simplify MLContext creation
anssik: PR #322
<ghurlbot> Pull Request 322 Simplify MLContext creation (wchao1115)
anssik: as a re-cap this PR proposes to remove MLDeviceType enum:
enum MLDeviceType {
"cpu",
"gpu"
};
anssik: and to remove "high-performance" from MLPowerPreference enum:
enum MLPowerPreference {
"default",
"high-performance",
"low-power"
};
anssik: this PR makes WebGPU a normative dependency for the WebNN API, streamlines the GPU acceleration path
… with these changes, GPU context can be created only from a WebGPU device
… in our CR plan of record we agreed that WebGPU interop needs more implementation experience
… as a WG we acknowledge WebGPU interop is very important to get right
… I want to give this important feature enough time to mature through implementation experience
… and thus I prefer not to rush this important feature into the initial CR release train
… I will suggest to add an appropriate text and a pointer to this change into the "Status of This Document" section of the upcoming CR to draw attention to this upcoming feature expected in a CR update
… then, we can coordinate with WebGPU contributors and the implementation for an appropriate time for landing this feature into a revised CR that is expected to follow the initial CR
… I think WebGPU folks' feedback and review is required and super important
dom: couple of remarks, pushing to a revised CR makes sense, was not clear on "v2" semantics
… do we understand the impact on implementations?
… implementations should start based on the CR?
… because it is not a minor change, having it on SOTD and also having it linked from the relevant parts of the spec too
anssi: v2 in the agenda is about revised CRs indeed
… which can be done relatively easily nowadays
… in terms of implementations - we're spec-ing and implementing together
… and implementors are directly involved in this conversation
… as we're proceeding to a recharter, I think it's important to show our progress with an initial CR
zkis: we're discussed a lot in PR - should some of these be moved to issues for greater visibility?
anssi: we could retroactively create an issue
zkis: having discussion hidden in PRs makes it harder to find from my experience
Rafael: I support Chai's proposal in the PR
… interop with WebGPU is important to move forward
… I think we should keep that flexibility open; not necessarily block on the WebGPU WG for progress
… e.g. for prototyping
… gaining implementation experience is key
<ghurlbot> Pull Request 322 Simplify MLContext creation (wchao1115)
PROPOSED: the initial CR will include an explicit call for implementation input on a simplify MLContext for WebGPU integration
PROPOSED: the initial CR will include an explicit call for implementation input on PR #322 to simplify MLContext creation
dom: the PR would be open for review and comments
… if we merge this PR, it would land in the next iteration of CR
anssik: my take is that we don't have enough implementation experience to ensure the model works
rafael: the current spec has the same needs of WebGPU interop & reviews
rafael: this PR improves the WebGPU interop from my perspective
anssi: we discussed marking the other piece of WebGPU interop as "at risk" given lack of implementation / review
… but this could change in a next CR
rafael: so would the current section on WebGPU remains in the spec with a warning?
anssi: right - or it could move to a separate spec if the group evaluates it's an improvement
rafael: I'm not sure why we shouldn't merge improvements to WebGPU interop given the lack of implementation experience overall in that space
anssi: if we merge this PR, the current implementation in chromium wouldn't reflect the spec
… the current impelemntation is partial but aligned with a subset of the spec
… with this PR, this would no longer be the case
https://
anssi: I see 2 options: keep WebGPU interop sections in the spec and mark them as needing more impl experience; or move them to a separate spec before the initial CR
dom: I'll chime in on process bits, we don't need implementation experience strictly to go to CR
… in that sense moving to a CR with or w/o PR merged is neutral
… the difference I see, because we are putting this in the code of the spec that is implementable without WebGPU support, it makes harder to argue we can move to CR without WebGPU WG review
… that is why we may want to stage the process in such a manner
anssi: we need to find a way to engage the WebGPU people on this review
ningxin_hu: this PR is orthogonal to WebGPU interop (which is done through MLCommandEncoder)
… it restructures the GPU context creation for WebGPU devices
… today's spec has two ways to create GPU context: with MLDeviceType and power preference setting, and a way from a GPUDevice
… this PR unifies the two into one and simplifies it
… based on my experience with implementations with existing backends, it actually makes it harder to implement as it removes flexibility
… keeping the two options allows to prototype and gain implementation experience across platforms
… there are risks we could hit blocking issues on some platforms
anssik: this reminds me of how WebRTC APIs depend on the same libwebrtc across browsers
… maybe not
… are there independent WebGPU implementations?
Rafael: 3 independent ones, yes
anssik: any thought on the challenge this may create e.g. with MLService on ChromeOS?
ningxin_hu: we need to address that gap for other ML native systems - it puts a requirement on native ML APIs and frameworks to support GPUs and need to recognize WebGPU devices and native implementations
… from my experience, this would be problematic at least on ChromeOS
… it's probably ok on windows, but we should check on MacOS and others - I suspect this may be too constraining there to adapt to these platforms
anssik: our charter states: "The APIs in scope of this group will not be tied to any particular platform and will be implementable on top of existing major platform APIs, such as Android Neural Networks API, Windows DirectML, and macOS/iOS Metal Performance Shaders and Basic Neural Network Subroutines." https://
ningxin_hu: cross-platform support is important for a Web API
dom: I'm hearing not so much about implementations, more about fundamental design question whether this design supports multiple platforms
… this is more a judgment call what we see as the most likely direction, will be hit a snag when doing x-platform implementation, or is the current design slowing adoption?
RafaelCintron: having multi-platform support is good - we should do that
… the editors have done a good job to ensure new operators are well supported across platforms and frameworks
… in MS, we definitely care about windows, but also very much about other platforms where we also have lots of stakes
… wrt putting in the spec vs moving to a different spec - the current spec refers to the current status of the WG work
… when it's iffy, it's ok to say so in the spec; it's also OK to move it to a separate document if that's better
… overall, I think the latest-and-greatest thinking should be merged in the spec, not languish in PRs
anssik: one trade-off is the lack of alignment between spec and implementation
… could we somehow show the two options as alternatives in the spec?
… e.g. with a note explaining the new model in the spec?
dom: this has been done in specs earlier, so not an issue for CR
dom: how certain we are that the new model would not be implementable on ChromeOS and other platforms?
dom: the level of applicability is important consideration
ningxin_hu: for ChromeOS, we would need more input from ChromeOS people
… from what we know in details today, the MLService backend doesn't interact with the WebGPU implementation - unclear if there is a path to a solution
… if we stick to the current context creation with MLDeviceType, we can support both ChromeOS and Windows in what we have today
… because the GPUDevice can be created and the MLContext can accept array buffers without interactions with WebGPU APIs
… if we simplify the MLContext creation to only GPUDevice, we can implement it in DirectML, but unclear for ChromeOS
dom: thanks! I'd say feasibility is complex space here due to platform, OS can evolve to what is required by this spec
… at the high level, to implement the change, if we think it through priority of constituencies
… easier API for developers is higher in priority
… OTOH, if implementers cannot adopt this API then it changes this picture
… I'm hearing, whether it is doable remains unclear, personal sense is maybe merging the PR is the right thing even we don't know if that leads to multi-platform implementations? Not pushing hard on one way or another. We may need to reverts change if impl exp proves otherwise.
anssi: our next milestone is the CR release to keep up with our commitment
… we need to converge on the content of the CR
… my initial proposal was to highlight the PR in the CR and annotate the WebGPU interop section in the spec - we didn't reach consensus, but that remains an option
… another option is to merge the PR (the impl is no longer in sync in that case), and we move to CR
… another option is to move WebGPU interop to a separate spec
… I like Rafael's point that we need to be as clear as possible in the body of the spec, put front and center our thinking, incl where we haven't converged and need more implementation experience
… quite a few options in front of us
RafaelCintron: if there is doubt about implementability, we should work it out in the PR
… the spec should reflect the group's thinking; annotating the spec when we lack implementation experience and we don't know if that will work is fine
… but it shouldn't be a presentation of disagreements per se
… I don't want CR to block us in getting the right content in the spec
… as for working with the WebGPU WG, on Feb 16-17, the WebGPU WG has a F2F in California - their first actual F2F since Covid
… the agenda of the F2F includes future directions of the API
… this would be a good opportunity to work with them on interop issues and expose more of what WebNN is and what it needs
anssi: thanks for sharing this, that's exciting!
… it would be great to have MLCommandEncoder presented there - e.g. if Chai was able to do so?
… or Ningxin remotely, if Chai can't
Anssi: I'll discuss with Dom on our path to CR as we discuss our rechartering
… heads up that we're hoping to move forward with our charter renewal