04:56:11 <RRSAgent> RRSAgent has joined #webmachinelearning
04:56:11 <RRSAgent> logging to https://www.w3.org/2022/03/09-webmachinelearning-irc
04:56:14 <Zakim> RRSAgent, make logs Public
04:56:14 <Zakim> please title this meeting ("meeting: ..."), anssik
04:56:16 <anssik> Meeting: WebML CG Teleconference – 9 Mar 2022
04:56:21 <anssik> Chair: Anssi
04:56:31 <anssik> Agenda: https://github.com/webmachinelearning/meetings/blob/master/telcons/2022-03-09-cg-agenda.md
04:56:31 <anssik> Scribe: Anssi
04:56:36 <anssik> scribeNick: anssik
04:56:41 <anssik> scribe+ Jonathan
04:56:47 <anssik> Present+ Anssi_Kostiainen
04:56:54 <anssik> RRSAgent, draft minutes
04:56:54 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
05:00:17 <Jonathan_Bingham> Jonathan_Bingham has joined #webmachinelearning
05:00:27 <anssik> Present+ Andrew_Moylan
05:00:39 <anssik> Present+ Jonathan_Bingham
05:00:41 <honglin> honglin has joined #webmachinelearning
05:00:45 <anssik> Present+ Honglin_Yu
05:01:01 <anssik> Present+ Joe_Bowser
05:01:17 <anssik> Present+ Jim_Pollock
05:02:06 <Andrew_Moylan> Andrew_Moylan has joined #webmachinelearning
05:02:17 <qjw> qjw has joined #webmachinelearning
05:03:03 <anssik> Present+ Shafron
05:03:07 <anssik> Present+ Qjw
05:03:27 <anssik> Present+ Tom_Shafron
05:03:30 <jim_pollock> jim_pollock has joined #webmachinelearning
05:04:07 <anssik> RRSAgent, draft minutes
05:04:07 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
05:04:53 <anssik> Present+ Sami_Kyostila
05:05:10 <Jonathan_Bingham> anssik: does the agenda look good?
05:05:16 <Sami> Sami has joined #webmachinelearning
05:05:21 <anssik> Topic: Model Loader API
05:05:22 <Jonathan_Bingham> ... [silence means yes ;-)]
05:05:28 <anssik> -> How to scribe https://github.com/webmachinelearning/meetings/blob/main/scribe-howto.md
05:05:35 <anssik> Subtopic: Versioning
05:06:11 <anssik> -> https://www.w3.org/2022/01/12-webmachinelearning-minutes.html#t02 minutes https://www.w3.org/2022/01/12-webmachinelearning-minutes.html
05:06:20 <Jonathan_Bingham> anssi: how to version the API for supported ops and formats was an open question from last time
05:06:55 <Jonathan_Bingham> honglin: we may need to support different versions of the backend
05:07:05 <Jonathan_Bingham> ... currently we're thinking of the version of TF Lite for the prototype
05:07:28 <Jonathan_Bingham> ... we're concerned about, in the future, if there are powerful new ops, but it has to be disabled, the version may not be enough
05:07:52 <Jonathan_Bingham> ... it may not be possible to fully support the backend, but it will need to be standardized for the clients
05:08:09 <Jonathan_Bingham> anssi: can you open a new github issue for this for async feedback?
05:08:21 <Jonathan_Bingham> ... not all of the stakeholders are here today
05:08:46 <Jonathan_Bingham> ... any rough sketch of a proposal?
05:09:09 <Jonathan_Bingham> ... it's briefly mentioned in the Explainer
05:09:16 <Jonathan_Bingham> q+
05:09:24 <anssik> ack Jonathan_Bingham
05:09:40 <anssik> Jonathan_Bingham: I think this topic is related to another agenda item, question on model formats
05:10:05 <anssik> ... in this prototype implementation TFLite flatbuf is the format and TF Runner an impl detail, the API abstracts over this backend
05:10:18 <anssik> ... perhaps the most natural way to do version is in terms of model format
05:10:29 <anssik> ... then question on op set a particular model supports, is that enough?
05:11:07 <anssik> q?
05:11:09 <Jonathan_Bingham> anssi: agree, let's connect the topics in the github issue
05:11:22 <anssik> Subtopic: Streaming inputs
05:12:02 <Jonathan_Bingham> honglin: this is also beyond the current state. for some APIs like translation or speech to text, the output might not be ready immediately
05:12:24 <Jonathan_Bingham> ... for speech recognition, you keep sending input. when the model thinks it's a complete sentence, it will give outputs
05:12:35 <Jonathan_Bingham> ... there may need to be a callback to the model, rather than the current interface
05:12:54 <Jonathan_Bingham> ... for now, we're not considering it. an API extension may be required
05:13:00 <anssik> q?
05:13:05 <qjw> q+
05:13:12 <anssik> ack qjw
05:13:36 <Jonathan_Bingham> jiewei: how is the speech recognition model implemented under the hood?
05:13:52 <Jonathan_Bingham> ... maybe the event should be handled by javascript rather than the model loader
05:14:30 <anssik> RRSAgent, draft minutes
05:14:30 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
05:14:34 <Jonathan_Bingham> honglin: that might be inefficient. for speech recognition in ChromeOS, the result is sent from the model spontaneously. it's not a TF Lite model in this example. but in the future TF Lite could support it.
05:14:48 <Jonathan_Bingham> ... maybe provide 2 functions: data feeding, and result accepting
05:14:57 <Jonathan_Bingham> ... the model will call it actively, by itself
05:15:00 <anssik> scribe+ Jonathan_Bingham
05:15:29 <Jonathan_Bingham> sami: it will be good to have concrete models to look at. it's not a blocker currently
05:15:57 <Jonathan_Bingham> (or was that Jiewei again?)
05:16:41 <anssik> q?
05:16:50 <Jonathan_Bingham> anssi: let's gather feedback in github again, outside of this call
05:17:03 <anssik> Subtopic: Model format
05:17:30 <Jonathan_Bingham> anssi: my perspective: thanks to Honglin and others' work, we're at the prototyping phase
05:17:45 <Jonathan_Bingham> ... my expectation is the implementation experience will help inform the standard model format discussion
05:18:12 <anssik> "The Model Loader API needs a standard format supported across browsers and devices for broad interoperability."
05:18:14 <Jonathan_Bingham> ... adoption of the model loader spec as a working group deliverable, or official standard, is conditional to agreeing on a standard format
05:18:18 <anssik> -> WebML WG Charter: Tentative Deliverables https://www.w3.org/2021/04/web-machine-learning-charter.html#tentative
05:18:38 <Jonathan_Bingham> q+
05:19:01 <Jonathan_Bingham> ... TF Lite format is the choice for experimentation. that's great. we have to start somewhere.
05:19:28 <Jonathan_Bingham> ... let's discuss the next steps to move toward a standard format.
05:19:33 <anssik> q?
05:19:55 <Jonathan_Bingham> ... what is the canonical reference for the TF Lite format used in the prototype?
05:20:11 <anssik> q?
05:20:13 <Jonathan_Bingham> ... is there a spec, or is the implementation the de facto description?
05:20:24 <anssik> ack Jonathan_Bingham
05:20:45 <anssik> Jonathan_Bingham: can provide link to TFLite flat buffer format as a reference
05:20:56 <anssik> ... that can probably contribute toward a standard
05:21:17 <anssik> ... I raised my hand for queue because I think Raphael from Msft would be a great person to summarize position from their side
05:21:50 <anssik> ... the developers needs a single format to work against and be confident it works across all devices and browser, developer ergonomics issue
05:21:59 <anssik> ... we at Google think the same
05:22:26 <anssik> ... then how we move forward from this prototype to something that could work for the wider community, a good topic, would love for Ningxin and Msft folks to chime in on this topi
05:22:30 <anssik> s/topi/topic
05:22:57 <anssik> ... proposal, let's look at Ningxin's tests for WebNN, he has looked at a few formats and tried to get them run on top of WebNN
05:23:16 <anssik> ... can we find a common intersection, starting point would be to explore what exists now
05:24:12 <anssik> ... I'm not suggesting Model Loader understands all those format, but that we can translate those formats into a format that Model Loader understands
05:24:45 <Jonathan_Bingham> anssi: that's a pragmatic approach. i agree it's about developer ergonomics.
05:24:52 <Jonathan_Bingham> ... licensing and other concerns we can figure out.
05:25:06 <Jonathan_Bingham> ... the purpose of standards is to have interoperability.
05:25:47 <Jonathan_Bingham> ... probably we'll never get to a world where there is one format only. we'll need tools to work between formats.
05:26:20 <Jonathan_Bingham> ... as with the previous topics, let's document in a github issue and get input from Ningxin and others.
05:26:47 <Jonathan_Bingham> ... we can follow up with Rafael and Ningxin
05:27:07 <Jonathan_Bingham> jonathan: i'll file the github issue
05:27:33 <anssik> q?
05:27:34 <Jonathan_Bingham> anssi: there's mention in the Explainer, but it will be easier to get discussion in an Issue
05:28:30 <anssik> q?
05:28:54 <anssik> Subtopic: Support for non-IETF 754 floating point types
05:29:01 <honglin> https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs
05:29:14 <anssik> -> Open question: support for non-IETF 754 float point types #23 https://github.com/webmachinelearning/model-loader/issues/23
05:30:14 <Jonathan_Bingham> jiewei: neither webnn nor model loader explains how to support it in their API
05:30:25 <Jonathan_Bingham> ... what's the plan?
05:30:29 <anssik> -> WebNN API: Support for non-IETF 754 float point types #252 https://github.com/webmachinelearning/webnn/issues/252
05:31:07 <anssik> -> Web & ML workshop discussion: Support for Float16 in JS & WASM environments https://github.com/w3c/machine-learning-workshop/issues/64
05:31:43 <Jonathan_Bingham> anssi: there was an earlier question for bfloat 16, which is mostly used for training accelerators
05:31:51 <Jonathan_Bingham> ... it's non standard currently and some concerns were raised
05:32:24 <Jonathan_Bingham> ... is this type also for training? does it improve inference performance too?
05:32:30 <Jonathan_Bingham> jiewei: that's an experiment we can do
05:32:42 <Jonathan_Bingham> ... i'm pretty sure it's not just for training, though it does improve training speed
05:33:04 <Jonathan_Bingham> ... it applies to inference, with dedicated accelerators. the speedup would be kind of the same.
05:33:18 <Jonathan_Bingham> ... it's worth experimenting whether the speedup is user-perceptible
05:33:50 <Jonathan_Bingham> anssi: how big of an effort would it be to generate and share this data?
05:34:03 <Jonathan_Bingham> jiewei: we might be able to do experiments with nvidia graphics cards
05:34:22 <Jonathan_Bingham> honglin: float16 can be easily supported, since the model loader accepts binary buffers
05:34:50 <Jonathan_Bingham> ... we want to have an idea of how webnn would support it, and then we can make it consistent with model loader
05:35:22 <Jonathan_Bingham> anssi: jiewei pointed out that the discussion is different depending on the API
05:35:32 <Jonathan_Bingham> ... for web nn, the discussion is how to use the types when building the graph
05:35:47 <Jonathan_Bingham> ... for model loader, it's about how to use the types during IO
05:35:58 <Jonathan_Bingham> honglin: in model loader, execution is internal. correct.
05:36:26 <Jonathan_Bingham> jiewei: if we want to interoperate between model loader and webnn, how do we pass the types between the two?
05:36:41 <Jonathan_Bingham> ... do we convert to a common format? or use a standard struct?
05:37:06 <Jonathan_Bingham> anssi: you proposed earlier that this could happen transparently, and auto-convert based on the accelerator. not sure if you'd get the performance benefit
05:37:25 <Jonathan_Bingham> jiewei: this applies to webnn; model loader can internally auto-convert
05:37:44 <Jonathan_Bingham> ... for webnn, the non-standard format would be for just part of the graph
05:38:06 <Jonathan_Bingham> anssi: you had another proposal about the API specifying acceptable quantization levels
05:38:42 <Jonathan_Bingham> jiewei: i can give an example of the improvement. if we quantize images with fp16, the power consumption can be reduced by 50% with performance the same
05:39:05 <Jonathan_Bingham> ... none of the APIs provide a way for developers to specify that
05:39:26 <Jonathan_Bingham> ... for some accelerators, it may provide a speedup. depends on the chips we're executing on.
05:39:37 <Jonathan_Bingham> anssi: what's the state of the prototype with respect to these types?
05:39:49 <Jonathan_Bingham> jiewei: it doesn't provide this option
05:40:21 <Jonathan_Bingham> anssi: we can validate the expected performance improvement, and that would motivate people on webnn to find a solution.
05:40:59 <Jonathan_Bingham> ... you can use the existing issues, and share data in the issue
05:41:15 <Jonathan_Bingham> ... i can. nudge the issue in the Working Group to get reactions
05:41:18 <anssik> q?
05:42:21 <anssik> Jonathan_Bingham: ChromeOS run on lower end devices where low power consumption is important, we probably don't have specific apps that are blocked on these types
05:43:29 <Jonathan_Bingham> anssik: would the extra types pose a privacy risk?
05:43:47 <Jonathan_Bingham> ... eg, distinguish between Apple's neural engine vs nvidia gpu
05:44:46 <Jonathan_Bingham> ... the first priority is to share data that the types would improve performance. the web gpu group is discussing similar things.
05:45:23 <anssik> RRSAgent, draft minutes
05:45:23 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
05:45:54 <anssik> Subtopic: WebNN API status update
05:46:05 <Jonathan_Bingham> honglin: what's the current state of webnn? our sydney team can't join the working group meeting
05:46:16 <Jonathan_Bingham> anssik: i'll share links.
05:46:56 <anssik> -> Intent to Prototype: Web Neural Network API (WebNN) https://groups.google.com/a/chromium.org/g/blink-dev/c/PD6TDMDS9mg/m/N3MrigMyCAAJ
05:47:19 <anssik> -> WebNN implementation in Chromium https://docs.google.com/document/d/1KDVuz38fx3SpLVdE8FzCCqASjFfOBXcJWj124jP7ZZ4/edit
05:47:52 <Jonathan_Bingham> ... the group has been focused on candidate recommendation spec. it's basically the feature complete state.
05:48:00 <Jonathan_Bingham> the target is to achieve it this year.
05:48:13 <anssik> -> WebNN API spec Candidate Recommendation readiness tracker https://github.com/webmachinelearning/webnn/issues/240
05:49:46 <anssik> ->  issues with a "cr" label https://github.com/webmachinelearning/webnn/labels/cr
05:51:05 <Jonathan_Bingham> ... other requirements to get to cr: we need a cross-browser test suite with web platform test (wpt), which chromium runs in continuous integration
05:51:25 <Jonathan_Bingham> ... the test suite confirms that the implementation conforms to the spec.
05:51:36 <Jonathan_Bingham> ... that's a major deliverable
05:51:53 <Jonathan_Bingham> ... the group has also started work on a pure javascript implementation without 3rd party dependencies
05:52:04 <anssik> -> Add WebNN baseline implementation for first-wave ops #1 https://github.com/webmachinelearning/webnn-baseline/pull/1
05:52:28 <anssik> -> WebNN-native https://github.com/webmachinelearning/webnn-native/
05:52:30 <Jonathan_Bingham> ... another project is webnn native
05:53:03 <Jonathan_Bingham> ... if you're familiar with webgpu implementations, it's like the webnn version of dawn
05:53:28 <Jonathan_Bingham> ... there's parallel work on scoping, polishing the implementation, wider review
05:53:56 <Jonathan_Bingham> honglin: because we're going to submit some CLs in a couple weeks, and we share some data structs with webnn, we may need to collaborate in the directory structure in blink
05:54:14 <Jonathan_Bingham> ... that's why i was asking about the status of webnn, because we want to coordinate
05:54:37 <Jonathan_Bingham> ... if they're going to submit later, we may just go ahead first
05:54:54 <Jonathan_Bingham> anssik: the aim is to have something behind a flag this year
05:55:26 <Jonathan_Bingham> ... that means it can still change
05:56:00 <Jonathan_Bingham> Jonathan_Bingham: are you going to start with a developer trial or go straight to an origin trial?
05:56:15 <Jonathan_Bingham> anssik: developer trial comes before the origin trial in the chromium launch process
05:56:35 <Jonathan_Bingham> ... we'll start with that, with a feature flag
05:56:57 <Jonathan_Bingham> ... the origin trial is for real production use with less tech-savvy developers
05:57:26 <Jonathan_Bingham> ... origin trials are not supposed to enable a feature for all users; if too many are using it, the trial will be disabled
05:57:37 <Jonathan_Bingham> ... developer trial is the goal for this eyar
05:58:22 <anssik> RRSAgent, draft minutes
05:58:22 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
05:59:12 <Jonathan_Bingham> Jonathan_Bingham: for model loader, dev trial is also the goal, starting very limited, with Chrome OS support only
05:59:29 <Jonathan_Bingham> ... for a wider dev trial, we'll want to get help from Chrome or TensorFlow
05:59:35 <Jonathan_Bingham> ... likely it will be late this year at the earliest
05:59:51 <Jonathan_Bingham> honglin: the chrome OS team has worked on the handwriting API before and has experience
06:00:15 <Jonathan_Bingham> anssik: our first origin trial for new sensor APIs was one of the first by non-Google people
06:00:23 <Jonathan_Bingham> ... this isn't our first attempt
06:00:58 <anssik> q?
06:01:21 <anssik> anssik: huge thanks Jonathan for scribing!
06:03:20 <anssik> RRSAgent, draft minutes
06:03:20 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
06:04:00 <anssik> s/Subtopic: WebNN API status update/Topic: WebNN API status update
06:04:07 <anssik> RRSAgent, draft minutes
06:04:08 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html anssik
06:32:55 <dom> dom has joined #webmachinelearning
07:15:33 <dom> i/Topic: Model Loader API/scribe+ Jonathan_Bingham
07:15:39 <dom> RRSAgent, draft minutes
07:15:39 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html dom
07:16:49 <dom> s|<honglin> https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs||
07:17:05 <dom> i|Subtopic: Support for non-IETF 754 floating point types|-> TFLite FlatBuffers format https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs
07:17:07 <dom> RRSAgent, draft minutes
07:17:07 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html dom
07:17:34 <dom> s|https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs||
07:17:35 <dom> RRSAgent, draft minutes
07:17:35 <RRSAgent> I have made the request to generate https://www.w3.org/2022/03/09-webmachinelearning-minutes.html dom
08:39:54 <dom> dom has joined #webmachinelearning
11:02:02 <dom> dom has joined #webmachinelearning
11:34:56 <dom> dom has joined #webmachinelearning
11:48:27 <Zakim> Zakim has left #webmachinelearning
12:56:16 <dom> dom has joined #webmachinelearning
13:19:14 <zkis> zkis has joined #webmachinelearning
13:33:29 <AramZS> AramZS has joined #webmachinelearning