Meeting minutes
Security review response
anssi: on the last call we reviewed the security review feedback from the chrome team
… and agreed that there was a lot of good feedback, with a few open questions from our end
… I started a draft pull request https://
… which I would like us to refine before circling back with the security people
PR: Update Security Considerations per review feedback
anssi: I need you review, please take a look at it
anssi: integrating that security feedback will be important in our path to Candidate Recommendation
Guidelines/philosophy for new operations, including security principles #242
anssik: I welcome feedback on my response to #242
Op metadata that helps avoid implementation mistakes #243
dom: I think this is about having pro-forma information about ways operations can create security risks
chai: IIRC, #243 was considered an implementation detail
chai: for some of the operations, you don't know the tensor shape before execution, when constructing the graph
… I don't see how you would be able to annotate / validate at construction time
… bound checking would have to be done at execution time
dom: I think the reviewer is asking us to highlight this information consistently across the spec, I think validation is not expected at graph construction time
… the comment says "create checks that can be enforced as a graph is constructed, and as a graph is executed"
ningxin_hu: some of the questions (e.g. tensor size input/output) have already answers in the prose
dom: my interpretation is, the reviewer is saying, to have this information consistently across ops, if provided in some regular convention, a subsection or table
… currently this is hidden in the prose, so having them extracted in a quick to parse format would facilitate reviews
ningxin_hu: is this metadata supposedly part of the spec, or a separate doc?
dom: in the spec, in addition to describing this information in prose have it in a consistent format e.g. in a table
chai: it may be not be possible to extract very useful metadata given the complexity of operations
anssik: let's clarify that in the spec and then with the reviewer
… chai, maybe you could chime in in the issue?
A conformance suite with disallowed intra-op examples would be helpful for hardening #244
anssik: any concerns with the proposal to formalize these failure cases in the test suite
<RafaelCintron> Anssi, I agree with your reponse in the Github issue.
chai: I'm not sure the notion of badly formed graphs applies here
… you can construct any graph you want; not sure what a badly formed graph would be
… I don't think we reject malformed graphs in DirectML
… one case perhaps would be an insanely complex graph that would trigger a DOS attack
ningxin_hu: would this be related to input buffers with out of bounds access (e.g. smaller buffer than what the graph defines)
… another scenario would be around constant uploading; the shape & uploaded buffer could not match and create another case of out of bound access
anssi: it would be helpful to bring this back to the issue
anssi: how would we bring this to WPT?
dom: how to bring this to w-p-t we need to discuss, failures would throw exceptions
… OOB would require us to be extra careful and thorough in covering error scenarios
<ningxin_hu> will do that
Privacy review refresh
anssi: we completed a first privacy review a year ago; we should discuss & identify changes that may impact privacy properties
… we would then go to PING with a delta change request
dom: yes that delta review sounds good
… there's a possibility PING understanding of privacy space has changed so they may also revisit some earlier discussions
… in terms of timing, given the discussion whether to make some part of the API device type specific, given the impact that may have on exposing what devices are available, my guess is it is better to wait for that to be clarified first
… only then do back to PING to not need to do back and forth too much
anssi: we should probably hand them the papertrail of the previous review to help
… also hearing the potential dependency on the device type
dom: if answers to questionnaire have changed, then revising those makes sense, give them diff from previous to the current and PRs that got merged with possible privacy impact
Double-precision baseline implementation of WebNN operations for testing
PR: WebNN baseline implementation for first-wave ops
GH repo (webnn-baseline is a CG deliverable)
WebML CG Charter: Test Suites and Other Software
ningxin_hu: this project is to answer the request to establish baseline results for the WebNN conformance test
… chai had mentioned this should have all the computation done in double precision, both for input & output data
… that this should be done in an open source implementation, and making it easy to review (dom helped improve readability)
… it was also suggested that reviewability would be helped by having very limited dependencies, by opposition to e.g. the polyfill where performance is more important
… So we developed this implementation that we want to contribute to the CG as a tool to generate results for the WebNN conformance tests
… We've implemented 41 operations linked to the 1st wave model, based on JS double precision numbers & calculation
… the algorithms are plain & straightforward algorithms without dependencies
… and brought it to a repository under the webmachinlearning github org, with the webnn-baseline name
… I think it's now in good shape for wider review of the implementation of operations
… to help with reviewability, input validation and other utilities have been moved to separate modules
anssi: thank you for bringing this work to that stage!
Rafael: is this all your code?
ningxin_hu: yes, all of our own
… to be clear, it's not a WebNN polyfill - it's implementing the WebNN ops
… based on their semantics, in JS
Rafael: can this be used along with the polyfill for a JS-only version of the spec?
ningxin: yes, but to the cost of performance
Rafael: I'm not opposed to the principle of having something like this
ningxin: a significant amount of code is test code to test the implementation
dom: just to highlight what ningxin_hu said, we as a WG should not be scared by the high LOC
… the idea is to be able to focus on the raw code of the computation
… if a few of us could give an indepth look on this, it'd increase our confidence further
… comparison with other implementation will allow us to compare the results
chai: we can help review conv2d
… for the other operators, there are quite straightforward
… we have our own baseline for our conformance testing, so we can do some comparison and see if you're missing some cases
… at a high level, the current code looks reasonable, but would want to make sure corner cases are taken care of
<ningxin_hu> appreciate
anssi: so let's keep the PR open to capture conv2d review from chai
<ningxin_hu> conv2d impl: https://
ningxin_hu: the complex ones like conv2d definitely require review; for other big ops (like gru and gruCell), we followed what's defined in the spec, by composing it on top of smaller ops
… we captured bugs in the spec while doing so
RESOLUTION: The WebML WG recommends its incubator WebML CG to adopt webnn-baseline as a new deliverable per "Test Suites and Other Software" provision to help demonstrate Web Neural Network API implementation experience
WebNN API open pull requests
Update createContext() and MLContext interface prose and algorithms #250
anssi: this might also help with the discussion around MLContext
https://
Add MLOperand and update MLOperator description #237
Fix output size calculation of conv2d and convTranspose2d #232
ningxin_hu: would like chai's review on the output size calculation due to the dilation parameter (based on impl feedback)
Adding a new use case for 'Framework Use Cases'
anssik: thanks everyone for joining and Dom for scribing!