Meeting minutes
WebNN API recent new feature requests
Request input layout and resize only on height and width for Resample
anssik: it seems this discussion is still active in PR #205 and we're converging on a slightly lower-level design that allows WebNN to act as a backend interface to ML frameworks most flexibly?
Chai: Ningxin proposed we'd use axis, want to hear from Rama how to use this properly?
Rama: (explaining the details of axis)
Chai: does axis work similarly in other ops too?
Rama: in most other ops yes, axis identification is straight-forward
Chai: in this case we know we're going to interpolate on the consecutive dimensions, you're saying them axis can be zero, not following the axis zero?
Rama: in ONNX it is zero-based, axis zero is allowed usually
… numpy convention allows zero and negative counts axis backwards from the last axis
<ningxin_hu> i understand the webnn is also zero based
Rama: zero-based and one-based is othogonal, you can have zero-based index
Chai: when zero, how do you describe it? interpolate in first two dimensions?
Rama: axis+1 are the dims to be scaled
… I guess I'm not sure I understand the question
… the axis in other ops are index 1
… in WebNN
… we need to define axis as zero-based to use the same notion across the API
… want to be consistent within this API and its ops
Chai: I'd look at slice for usage, there are other ops as well
… slice is a good example, the same semantics
Rama: OK, using 1-based should be fine then, I'm a bit surprised still
<ningxin_hu> for slice, the axes values in the sequence are either within the [0, r-1] range where r is the input tensor rank
<ningxin_hu> https://
Chai: for example, we interporlate the last two dims, axis equal 2, we interpolate along the second dimension
… how to define axis when interpolating along the first dim?
Rama: we can continue discuss this in the GH issue
… higher-level question: choosing this design, considering automation in other frameworks, people are writing code manually, it is helpful to have an API that is constrained?
… a more generous API may make this simpler perhaps
Chai: this was a point I was trying to make, when you design the API, normally from the get go you want to ask whether you optimize for ease of use, that web dev can call directly
… e.g. give enums
… that's not the first goal, we're targeting ML frameworks and this is backend API sitting underneath
… interface should be flexible enough, but flexible enough
… e.g. ONNX has no layout or axis for resize op
… when converting from ONNX to WebNN, whether from tool or at runtime, it should be possible
ningxin_hu: thanks for the discussion, very helpful!
… I agree with Chai that the WebNN API key customer is an ML JS Framework
… and e.g. converting from ONNX should be possible
… the previous sizes and scales are good design, most flexible
… the only issue raised as an implementation feedback was that the previous spec was not clear what cases the interface supports, e.g. more than 2 dims interpolation
… also could be scattered
… implementation feedback suggests it is hard to map this flexible interface to the underlying ML APIs, e.g. OpenVINO and NNAPI
… Chai just made it clear this API is sometimes limited, and we can go back and put the limitation that this interface supports three cases
… I think implementation could handle that
Chai: want to add, agree with Ningxin
… mostly we agree on the details
… this should be 2D op and the proposed name change makes sense
… even ONNX is very open ended but does not do volumetrics, DirectML can do that
… interpolation makes a lot of sense to me
… limiting scale and sizes to be 2D makes sense, just need to distinguish the three separate cases mentioned in the GH issue:
[x, x, 1, 1]
[1, x, x, 1]
[1, 1, x, x]
ningxin_hu: different solution to axis proposal from Rama is to allow the developer to define two consecutive dims as spatial dims
… if the axis is zero-based
… to extend Rama's proposal, for simple axis sequence of two, the developer can define two consecutive dims as spatial
… that's a quick thought, not sure if perfect, but came to my mind just listening to this discussion
<ningxin_hu> sgtm
anssik: proposal for Ningxin to hash out the design in PR #205 and seek review from Chai, Rama -- we seem to be close to consensus
Support for configuring rounding type in pooling operations
anssik: Chai suggested adding an optional output size's rounding model
… does this address the issue? Ningxin you said SGTM? Does this work for OpenCV?
Chai: DirectML is an API that relies on the caller to spec the output size
… DML would not calculate the size
… in that case you'd have to do all the work
… works from DML, since frameworks sit on top
… the questions about DML helper lib, can dig an answer to that
… not super easy to use DML because it is not a goal, DML does not need to be easy but explicit
… mapping DML to WebNN would be nice but it does not have to map 1:1, the frameworks like TF or ONNXRuntime the backend has to do shared inference
… that's probably the difference there
ningxin_hu: your proposal to add output sizes SGTM, framework calc the output size and ignore the rounding mode
… OpenCV relies on backend to calc the size, so good to use a rounding model to map that and backend can help calc that with WebNN implementation
… if we'd have these two it'd be most flexible
… I think this is similar to the Conv2D, we have explicit padding
… I agree with Chai's proposal and will update the PR accordingly
… second questions: DML design, explicit output size, DirectML X, in WebNN implementation is based on DirectML X that is easier to use
… question about DML X pooling op, its graph builder API similar to WebNN
… caller set the output size, but in DML X pooling API it can't provide any output size or rounding model
… avg pooling calc the output size, but interface to DML X does not have the capability to set rounding model
… interested in how DML X supports this feature
Chai: DML itself is also a graph API, DML X does not add new features, makes it easier to use
… WebNN can defer all this to runtime, but DML X does not do that, will help calculate, but you have to lay it out when constructing the graph
ningxin_hu: I'll look at DML X implementation and will follow up
ningxin_hu: will add output size per Chai's suggestion to PR #208
reduceLogSum, reduceLogSumExp, reduceSumSquare are not supported on OV/NNAPI
anssik: the issue is brief, states reduceLogSum, reduceLogSumExp, reduceSumSquare are not supported by OpenVINO or NNAPI
… I believe the intent of this issue is to probably discuss whether those three ops should stay in the spec?
… would emulating these ops on those platforms be an option?
<Chai> it looks like all of them can be emulated
<rama> I think so too. They are just compositions of other ops.
<Chai> i can take an action item to check it out.
Chai: I'll take a look and confirm
anssik: it looks like we may want to add an informative section on emulation
… for these three ops
[webidl] Add method steps to operations
anssik: The WebIDL spec recommends algorithmic method steps for defining operations: https://
… This spec currently uses a WebGPU spec inspired convention that differs from this best practice. We should consider adopting the "method steps" convention.
… An example of method steps in context of the WebNN API spec:
https://
I believe @domenic would be happy to review the PR for this, perhaps also Dom
[webidl] Define algorithms for dictionaries with lists as default values
anssik: quoting domenic:
… You need to have a normative algorithm which checks if input["scales"]
… exists, and if not, use the defaults. I.e. something like
… 1. Let scales be the list « 1.0, 1.0, 1.0, 1.0 ».
… 2. If input["scales"] exists, then set scales to input["scales"].
… (and maybe also validate that input["scales"] is of length 4, or whatever you need?)
… This seems difficult since right now your spec has no algorithms (i.e. no method steps) for its operations, just a non-normative description of the inputs and return value.
anssik: My suggestion was to make this a global update to all affected dictionaries. We probably want to define a reusable algorithm we reference from all these places.
[usecase] Neural network deployment
anssik: Sungpil Shin submitted this use case for consideration.
… I reviewed this and had a question whether this use case is similar to https://
… we could consider merging the two perhaps?
Sungpil_Shin: thanks Anssi, we suggested this PR with Wonsuk Lee
… we reviewed your comments, and agree the point of this use case is quite similar to the other use case already in the spec
… but our point for this use case is to select the best hardware in devices
… device selection discussion seem to have touched on this use case?
… we will revise this use case
https://
Chai: this seems different from issue #169
… Sungpil is proposing something new, allow caller to select on which device to run time one, this seems to be a new capability, essentially we could consider adding this new use case if this is what we think WebNN should be able to do
… this happens automatically, someone has a model, converted to WebNN, expected to run on any hardware without specifying explicitly on which devices it'd run on
… this automation that'd be required in the API is way more than selecting what devices are available, would involve analyzing the model, e.g. there are models that are more effective to be run on CPU and in some cases on GPU e.g. CV models
… this use case basically says WebNN has to choose, beyond the scope we have now
Sungpil: focused on learning rate to find the best hardware in this use case
TPAC agenda building
anssik: Call to review the draft agenda for the Web Machine Learning WG Virtual Meeting at TPAC 2021
WebML WG Virtual Meetings at TPAC 2021
anssik: We'd like to get your proposals in as soon as possible so we can split topics across meetings days matching folks interests
anssik: poll was not very well attended, Rafael is not able to join on 26 Oct, I assume for other participants the date are fine.
… these proposed dates are now official:
26 October 2021 14:00-15:00 UTC+0
27 October 2021 14:00-15:00 UTC+0
28 October 2021 14:00-15:00 UTC+0