03:56:43 RRSAgent has joined #webmachinelearning 03:56:43 logging to https://www.w3.org/2022/05/04-webmachinelearning-irc 03:56:46 RRSAgent, make logs Public 03:56:46 please title this meeting ("meeting: ..."), anssik 03:57:02 Meeting: WebML CG Teleconference – 4 May 2022 03:57:18 Chair: Anssi 03:57:31 Agenda: https://github.com/webmachinelearning/meetings/blob/main/telcons/2022-05-04-cg-agenda.md 03:57:46 Scribe: Anssi 03:57:55 scribeNick: anssik 03:58:04 scribe+ Jonathan 03:58:19 Present+ Anssi_Kostiainen 03:58:57 RRSAgent, draft minutes 03:58:57 I have made the request to generate https://www.w3.org/2022/05/04-webmachinelearning-minutes.html anssik 04:00:52 Honglin has joined #webmachinelearning 04:02:38 Geun-Hyung has joined #webmachinelearning 04:04:08 RafaelCintron has joined #webmachinelearning 04:04:48 Joe_Bowser has joined #webmachinelearning 04:05:28 Present+ Honglin_Yu 04:05:37 Present+ Chai_Chaoweeraprasit 04:06:02 Present+ Geun_Hyung_Kim 04:06:31 Present+ Chai_Chaoweeraprasit 04:06:39 Present+ Joe_Bowser 04:06:56 Present+ Rafael_Cintron 04:07:23 RRSAgent, draft minutes 04:07:23 I have made the request to generate https://www.w3.org/2022/05/04-webmachinelearning-minutes.html anssik 04:08:02 anssik: Goal for the first topic is to discuss and triage new issues; identify any reusable constructs, concepts and patterns to be shared with WebNN API. 04:08:10 Subtopic: Device Preferences 04:08:37 anssik: three issues around a common theme of device preferences, touched on 6 Apr meeting briefly 04:08:45 -> WebML CG call 6 Apr 2022: Device Preferences: GPU, power, usage, performance hints https://www.w3.org/2022/04/06-webmachinelearning-minutes.html#t05 04:08:52 -> DevicePreference: "GPU" => "GpuRequired" and "GpuPreferred" https://github.com/webmachinelearning/model-loader/issues/35 04:08:58 -> "default" v.s. "auto" in MLDevicePreference and MLPowerPreference https://github.com/webmachinelearning/model-loader/issues/30 04:09:06 -> Add performance/usage Hints https://github.com/webmachinelearning/model-loader/issues/34 04:09:52 anssik: from that discussion, it seems there's interest to try to share surface between WebNN and Model Loader 04:11:07 ... Rafael from Msft noted "auto" has precedent, helps mitigate fingerprinting concerns, the browser to pick the backend for better of worse (i.e. selection is an implementation detail) 04:11:40 Present+ Raviraj_Pinnamaraju 04:12:32 https://github.com/webmachinelearning/model-loader/issues/30 04:13:15 chai has joined #webmachinelearning 04:13:38 Rafael: we've been worn many times with an API that gives a bunch of devices 04:13:50 ... better to say, "browser give me an ML device" 04:14:19 ... for WebXR they go further, have required and optional hints 04:15:09 anssik: any experiences from WebXR, does their API work, interop issues? 04:15:38 Rafael: people tend to make assumptions sometimes, no major issues known with their approach 04:16:01 ... earlier WebVR API was worse, that required ppl to do string comparison etc. 04:16:29 ... getUserMedia and WebXR models are best for privacy 04:16:50 Honglin: maybe adapting the model to the device is not a problem in terms of privacy? 04:17:20 ... exposing GPU specific features, would that be a problem? 04:17:40 ... we don't want to leak device information, e.g. what type of GPU is in use 04:17:59 ... I think it is not an issue, it is already known what the GPU is 04:18:24 Rafael: reasonable to expose an API that prefers GPU, WebGL supports PowerPreference, not a guarantee a wish 04:18:45 ... in Chromium, if you has high-power, you actually never get such a battery-sucking context 04:19:23 ... low-power, high-power, (pick nothing) are the options 04:19:35 ... in iframe, high-power does nothing 04:20:45 also "fail is major perf caveats" exists, example "Google Maps" 04:21:01 Honglin: is this the same with WebGPU? 04:21:17 ... does this affect WebGPU backend of WebNN? 04:21:36 Rafael: I don't know we ignore that for WebNN 04:21:55 Chai: for WebNN, you can create WebNN context either from WebGPU device or its own device 04:22:06 ... to interop with WebGPU, you pass the context to WebNN 04:22:30 ... standalone WebNN usage is possible, then can provide preferences 04:23:15 Honglin: this reminds me of an issue, the model is running in another process, more complex power management, e.g. when the renderer is throttled, we probably need to also throttle the other process 04:23:36 Rafael: what do you do when multiple web pages ask of different things, process per request? 04:23:47 ... for power management, for example 04:24:03 https://immersive-web.github.io/webxr/#feature-dependencies has a good explanation of required vs. optional features for WebXR. 04:24:09 Honglin: haven't gone to the details, talking with experts the plan 04:24:42 ... TFLine has a callback, maybe we can sleep in between, this is being explored 04:25:13 Rafael: when ppl say throttling it doesn't mean slower always, timeouts are fired less frequently 04:25:41 ... when you minimize a tab, Chrome will constrain the CPU usage of the tab 04:25:59 RRSAgent, draft minutes 04:25:59 I have made the request to generate https://www.w3.org/2022/05/04-webmachinelearning-minutes.html anssik 04:26:51 Rafael: TFLite in renderer process? what's limiting that? 04:27:38 Honglin: TGLite not designed for web usage, adding fuzzers to TFLite as a pending task, first is security, TFLite is also evolving quickly, so maybe not ready for renderer 04:27:43 https://www.khronos.org/registry/webgl/specs/latest/1.0/#5.2.1 has a good description of context parameters you can pass for requesting power preference and "failIfMajorPerformanceCaveat". 04:28:17 ... the renderer has its own memory management, security sandbox etc. 04:28:35 Rafael: evolving quickly, can you clarify? 04:29:21 Honglin: I don't think any ML packages are stable, they are evolving quickly 04:30:27 Chai: in my day job we work with ISVs and ML across hardware vendors there's significant overlap so we can build an abstract API atop 04:30:44 ... if we look at transformers, yes, that is an area that evolves quickly 04:31:11 ... for the most part, e.g. Computer Vision is settled, a lot of models there 04:31:32 ... significant overlap what the web can take advantage of 04:31:50 ... we can add stuff and break some stuff along the way 04:32:11 ... we should look at native, web is very basic and room to catch up there 04:33:22 Honglin: many technologies used to implement ML capabilities are evolving, ML foundations are stable I agree 04:34:27 ... what we choose now is the lowest effort approach, fail quickly and course correct to gather developer feedback 04:35:11 ... Chrome OS has a lot of users so good platform to gather feedback 04:35:34 ... I'm working on GPU support for Model Loader API currently 04:35:54 Chai: stable vs unstable, I meant the platform API is stable, not models 04:36:39 ... in the beginning of browsers, they ran on platform APIs, Mac, PC, browser has been successful abstracting the capabilities of the OS platforms, e.g. 2D Canvas 04:37:27 ... 2D graphics was developed enough when 2D Canvas was brought to the web, and it became successful 04:37:40 ... we can draw parallels to current platform APIs for ML 04:38:21 ... with native support in the OS, the OS can do better job in performance, hw conformance, because OS teams work with HW vendors, and web should use these HW capabilities 04:39:06 ... for example, CoreML does conv, matmul etc. similarly to Windows, Linux, others 04:39:41 ... conformance work done in the OS can benefit the web, portability is part of the Web API promise 04:40:28 ... companies continue invest in platform APIs is a clear signal that the browser should take advantage of these platform APIs, regardless of what is the next shiny object 04:42:18 anssik: there's also precedent in HTML media, canPlayType() method that must return "probably" if the user agent is confident that the type represents a media resource that it can render if used in with this audio or video element; and it must return "maybe" otherwise. 04:42:29 ... this may not be the greatest API, but it is out there and widely supported 04:42:46 -> videoElement.canPlayType() method from HTML spec https://html.spec.whatwg.org/multipage/media.html#dom-navigator-canplaytype 04:43:12 ... we could evaluate whether a similar "canComputeModel()" method would work here 04:44:16 q+ 04:44:22 ack RafaelCintron 04:45:21 Rafael: my understanding is that after you pick your preference any model would work 04:46:15 ... now if we'd target a large diversity of HW, and some ops wouldn't work at all, I think a better to have a set of ops for which you could say "yes I can do that" 04:47:28 Chai: I don't have many thought yet on this 04:48:05 Subtopic: Versioning 04:48:14 anssik: we merged this issue into Standard model format issue https://github.com/webmachinelearning/model-loader/issues/28 per https://www.w3.org/2022/03/09-webmachinelearning-minutes.html#t02 04:48:22 ... any concerns? 04:48:40 Honglin: Jonathan should own this issue 04:48:57 Subtopic: Standard model format 04:49:21 -> Standard model format https://github.com/webmachinelearning/model-loader/issues/28 04:49:32 -> TFLite FlatBuffers format https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/schema/schema.fbs 04:49:39 -> TFLite FlatBuffers format (commit history) https://github.com/tensorflow/tensorflow/commits/master/tensorflow/lite/schema/schema.fbs 04:49:56 anssik: I have one question, what is the process by which the TFLite FlatBuffers format is evolved? 04:50:12 ... for example, new features landing recently to add RELU_0_TO_1, support uint16 04:50:57 Honglin: I don't know exactly, TFLite ppl need to speak for it 04:51:24 ... I'm interested because of the provision we have in the WebML WG charter: 04:51:34 Note: The Model Loader API needs a standard format supported across browsers and devices for broad interoperability. The Working Group will only create a standard format of its own if no other standardized format aligns with the group's principles nor allows the group to control or provide direct input to shape it. The Working Group will only start working on this API when there is agreement on such a format. 04:51:43 https://www.w3.org/2021/04/web-machine-learning-charter.html#tentative 04:51:49 ... "The Working Group will only create a standard format of its own if no other standardized format [...] allows the group to control or provide direct input to shape it" 04:52:21 ... is the process by which TFLite FlatBuffers format is evolved community-driven and accepts direct input? 04:53:40 Rafael: not sure, but my bet is maybe no, it is a Google thing, the same goes with ONNX Runtime, where one format is a standard format, but it also support a proprietary format that mirrors ONNX Runtime 04:54:02 ... we should at least have a standard format every browser can adapt 04:54:45 RRSAgent, draft minutes 04:54:45 I have made the request to generate https://www.w3.org/2022/05/04-webmachinelearning-minutes.html anssik 05:10:32 s/... I'm interested/anssik: I'm interested 05:10:34 RRSAgent, draft minutes 05:10:34 I have made the request to generate https://www.w3.org/2022/05/04-webmachinelearning-minutes.html anssik 06:19:16 qjw has joined #webmachinelearning 06:27:34 qjw has joined #webmachinelearning 07:01:17 zkis has joined #webmachinelearning 09:03:28 Zakim has left #webmachinelearning