14:54:11 <RRSAgent> RRSAgent has joined #web-hybrid-ai
14:54:15 <RRSAgent> logging to https://www.w3.org/2024/09/25-web-hybrid-ai-irc
14:54:15 <tpac-breakout-bot> RRSAgent, do not leave
14:54:16 <tpac-breakout-bot> RRSAgent, make logs public
14:54:17 <tpac-breakout-bot> Meeting: Web API for Hybrid AI
14:54:17 <tpac-breakout-bot> Chair: Chunhui Mo, Martin Alvarez-Espinar
14:54:17 <tpac-breakout-bot> Agenda: https://github.com/w3c/tpac2024-breakouts/issues/44
14:54:17 <Zakim> Zakim has joined #web-hybrid-ai
14:54:18 <tpac-breakout-bot> Zakim, clear agenda
14:54:18 <Zakim> agenda cleared
14:54:18 <tpac-breakout-bot> Zakim, agenda+ Pick a scribe
14:54:19 <Zakim> agendum 1 added
14:54:19 <tpac-breakout-bot> Zakim, agenda+ Reminders: code of conduct, health policies, recorded session policy
14:54:20 <Zakim> agendum 2 added
14:54:20 <tpac-breakout-bot> Zakim, agenda+ Goal of this session
14:54:20 <Zakim> agendum 3 added
14:54:20 <tpac-breakout-bot> Zakim, agenda+ Discussion
14:54:22 <Zakim> agendum 4 added
14:54:22 <tpac-breakout-bot> Zakim, agenda+ Next steps / where discussion continues
14:54:22 <Zakim> agendum 5 added
14:54:22 <tpac-breakout-bot> tpac-breakout-bot has left #web-hybrid-ai
15:32:58 <Lei_Zhao8> Lei_Zhao8 has joined #web-hybrid-ai
15:34:33 <Lei_Zhao> Lei_Zhao has joined #web-hybrid-ai
16:36:55 <martin> martin has joined #web-hybrid-ai
16:37:55 <reillyg> reillyg has joined #web-hybrid-ai
16:49:40 <martin> martin has joined #web-hybrid-ai
16:50:43 <martin> martin has joined #web-hybrid-ai
17:00:04 <martin> Slides for the breakout session: https://kevinmoch.github.io/web-hybrid-ai/Templates/Overview.html
17:00:12 <martin> present+
17:00:22 <xfq> xfq has joined #web-hybrid-ai
17:02:13 <xfq> present+
17:02:31 <xfq> scribe: xfq
17:02:33 <QingAn> QingAn has joined #web-hybrid-ai
17:02:42 <xfq> martin: my name is Martin
17:02:54 <kenji_baheux> kenji_baheux has joined #web-hybrid-ai
17:02:55 <riju3> riju3 has joined #web-hybrid-ai
17:03:11 <McCool> McCool has joined #web-hybrid-ai
17:03:14 <dezell> dezell has joined #web-hybrid-ai
17:03:15 <xfq> ... my colleague from Huawei, Chunhui Mo, will present this breakout session for this new API
17:03:16 <xiaoqian> xiaoqian has joined #web-hybrid-ai
17:03:25 <dezell> present+
17:03:30 <sisidovski> sisidovski has joined #web-hybrid-ai
17:03:35 <asully> asully has joined #web-hybrid-ai
17:03:35 <kenji_baheux> present+
17:03:35 <ora> ora has joined #web-hybrid-ai
17:03:37 <xiaoqian> present+
17:03:40 <andrewnolan> andrewnolan has joined #web-hybrid-ai
17:03:53 <ora> present+
17:03:57 <kenji_baheux> can someone repost the link?
17:03:58 <Sarah> Sarah has joined #web-hybrid-ai
17:04:08 <xfq> ... I already shared the slides with you in the Zoom chat
17:04:14 <xfq> https://kevinmoch.github.io/web-hybrid-ai/Templates/Overview.html
17:04:21 <JRJurman> JRJurman has joined #web-hybrid-ai
17:04:32 <kenji_baheux> appreciated!
17:04:46 <xfq> chunhui: I'm Chunhui Mo
17:05:04 <xfq> ... the main topic is the new web API proposal to support hybrid AI development
17:05:05 <lambda> lambda has joined #web-hybrid-ai
17:05:23 <xfq> ... enable web developer to directly access on-device and cloud-based models
17:05:40 <xfq> ... and safely sharing user data across multiple apps when using these models
17:06:02 <xfq> ... an example is @@
17:06:13 <xfq> ... since the AI topic covers a broad range of areas
17:06:27 <xfq> ... I would like to clarify what is not in the scope
17:06:30 <martin> Repository with the proposal: https://github.com/kevinmoch/web-hybrid-ai
17:06:32 <ora> ora has joined #web-hybrid-ai
17:07:02 <xfq> ... 1. Designing a uniform JavaScript API for accessing browser-provided language models, which is currently being explored by Chrome's AI team
17:07:13 <xfq> ... known as the Prompt API
17:07:35 <xfq> ... 2. Issues faced by hybrid AI, already covered by other breakout sessions
17:07:42 <Lei_Zhao> present+
17:07:45 <xfq> ... let's dive into Connection API
17:08:18 <xfq> ... the prompt API @@
17:08:26 <xfq> ... both on-device and cloud-based
17:08:50 <xfq> ... now let's talk about the switch model
17:09:03 <xfq> ... switch between different apps
17:09:06 <martin> s/the prompt API @@/This is an extension of the Prompt API
17:09:15 <xfq> ... similar to connect
17:09:25 <xfq> ... this method is also an extension of the prompt API
17:09:37 <ktoumura> ktoumura has joined #web-hybrid-ai
17:10:04 <edward> edward has joined #web-hybrid-ai
17:10:06 <xfq> chunhui: now let's look at the sample code
17:10:22 <xfq> [Chunhui shows the sample code]
17:11:09 <xfq> ... I will give you a brief review of the methods
17:11:16 <xfq> rrsagent, make log public
17:11:19 <xfq> rrsagent, make minutes
17:11:20 <RRSAgent> I have made the request to generate https://www.w3.org/2024/09/25-web-hybrid-ai-minutes.html xfq
17:11:33 <xfq> s/... I will give you/chunhui: I will give you
17:12:02 <xfq> chunhui: now I'll cover the Storage API
17:12:54 <xfq> [Chunhui shows Storage API Sample Usage]
17:13:47 <xfq> chunhui: 'entry' is any kind of text including the user's private data
17:14:22 <xfq> ... we store the info under the user's profile, in the travel category
17:14:35 <xfq> ... if we have a web app connecting to the on-device model
17:15:14 <xfq> [Chunhui shows Storage API Implementation References]
17:16:13 <xueyuan> xueyuan has joined #web-hybrid-ai
17:16:57 <xfq> Chunhui: let's see how we can combine the Connection and the Storage API
17:17:14 <xfq> ... imagine we have planned a travel for upcoming holidays
17:17:22 <xfq> ... first we open a flight booking app
17:17:33 <xfq> ... and search for our destination
17:17:40 <xfq> ... this app uses a cloud-based AI model
17:18:11 <xfq> ... the recommend the best flight based on our needs
17:18:24 <xfq> ... next maybe open a hotel booking app to find a place to stay
17:18:31 <xfq> ... as you can see in this diagram
17:18:44 <xfq> ... the architecture for the flight and hotel apps
17:18:53 <xfq> ... can be broken down into the following steps
17:19:02 <tomayac> tomayac has joined #web-hybrid-ai
17:19:19 <xfq> ... assistants gather the user information/preference
17:19:29 <xfq> ... and send a request to the cloud-based model
17:19:35 <xfq> ... and retrieve the flight options
17:19:51 <xfq> ... once the flight information is returned
17:20:07 <xfq> ... the system uses the Storage API to convert the date
17:20:24 <xfq> ... stored into the database
17:20:36 <xfq> ... @@
17:20:53 <xfq> ... finally @@ will be sent to the cloud-based model
17:21:07 <xfq> [Chunhui shows a demo]
17:21:47 <xfq> chunhui: Considerations for Connection Strategy
17:22:01 <xfq> ... when we're talking about building apps for the hybrid AI
17:22:22 <xfq> ... there are several crucial elements that need to be considered
17:22:30 <xfq> ... 1. the connection time
17:22:46 <xfq> ... we need to think about how the system connect to different AI models, for instance
17:23:01 <xfq> ... if one model takes too long to respond
17:23:07 <xfq> ... we can skip it
17:23:16 <xfq> ... Custom Connection Strategy
17:23:28 <xfq> ... give developer flexibility
17:23:48 <xfq> ... we should offer more dynamic and efficient solution
17:23:53 <xfq> ... Model Status Information
17:24:04 <xfq> ... knowing the status of each model is key
17:24:12 <xfq> ... is a model available?
17:24:24 <xfq> ... Load Balancing Across Models
17:24:39 <xfq> ... helps us automatically distribute requests
17:24:46 <xfq> ... version control
17:24:57 <xfq> ... Considerations for Storage Strategy
17:25:06 <jfernandez> jfernandez has joined #web-hybrid-ai
17:25:19 <xfq> ... @@
17:25:39 <xfq> ... now let's look at a step further by looking at Considerations for Native OS APIs
17:26:09 <xfq> ... need to ensure that the AI model works consistently no matter which device is being used
17:26:25 <xfq> ... by leveraging the OS
17:26:42 <xfq> ... reduce overhead that comes with browser-based storage
17:27:09 <xfq> ... allowing the system to store more data and handle complex operations, which is fairly useful
17:27:18 <xfq> ... when working with large AI models
17:27:20 <McCool> q+
17:27:28 <kenji_baheux> q+
17:27:34 <reillyg> q+
17:27:35 <xfq> ... exploring the native OS integrations
17:27:54 <xfq> ... now we're moving into the discussion phase
17:27:54 <martin> ack McCool
17:28:03 <xfq> ... I'd love to hear your thoughts and feedback
17:28:08 <xfq> ... how we move forward
17:28:31 <xfq> McCool: you spent a lot of time discussing how to enable sharing data between apps
17:28:35 <RobKochman> RobKochman has joined #web-hybrid-ai
17:28:48 <Domenic> Domenic has joined #web-hybrid-ai
17:28:50 <xfq> ... however big concern the web is controlling the sharing the data bewteen apps
17:29:05 <martin> q?
17:29:05 <xfq> ... you don't wan to give all apps access all personal data
17:29:26 <xueyuan> s/wan/want/
17:29:32 <xfq> chunhui: we strore the data in the @@
17:30:11 <xfq> McCool: in one of our enterprise applications you can do access controls for accessing certain subsets of data
17:30:13 <ora> ora has joined #web-hybrid-ai
17:30:37 <xfq> ... having a mechanism for access control or you can explicitly grant some data
17:30:40 <martin> ack kenji
17:31:18 <xfq> kenji_baheux: i was wondering if you could share some of the details of the trigger of this exploration
17:31:26 <xfq> ... like the use cases
17:31:59 <kenji_baheux> concrete examples of use cases (internal) or from actual partners?
17:33:06 <xfq> chunhui: the use cases is 2 different apps not directly share the same data, but into the same local database
17:33:36 <xfq> q?
17:33:45 <martin> ack reil
17:33:54 <zolkis> zolkis has joined #web-hybrid-ai
17:34:05 <xfq> reillyg: to understand the context here, this is a proposal that would be exposed by the browsers?
17:34:09 <xfq> chunhui: yes
17:34:25 <xfq> reillyg: the API will allow multiple sites to insert data into this database
17:34:41 <xfq> ... and they can prompt the AI to user the data @@
17:35:08 <zolkis> q+
17:35:46 <xfq> ... the results from the AI are visible to the apps making the query
17:35:50 <kenji_baheux> q+
17:36:05 <xfq> ... if the AI uses information about travel, the app can see the response from the AI
17:36:19 <xfq> chunhui: if the apps have permisson, yes, they can
17:36:28 <xfq> ... they can benefit from the whole database
17:36:53 <xfq> reillyg: granting the safe access to their own device model to this database?
17:37:08 <Domenic> q+
17:37:14 <xfq> chunhui: another impl to implement the insert/update entries
17:37:22 <xfq> ... for your reference
17:37:54 <McCool> q?
17:38:05 <martin> ack zol
17:38:10 <xfq> zolkis: I understand that you want to build this solution
17:38:28 <xfq> ... is it mandatory requirement that you have booking and flight in separate apps?
17:38:45 <xfq> ... if you can do it in the same app, then you @@
17:39:03 <xfq> ... basically you want a web app
17:39:14 <martin> q?
17:39:14 <McCool> q?
17:39:15 <xfq> chunhui: let's say the Chrome prompt API
17:39:33 <tomayac> q+
17:39:52 <xfq> ... web devs use window.ai to connect LLM
17:40:05 <xfq> ... it's just extension for Chrome's prompt API
17:40:16 <McCool> q+
17:40:22 <xfq> ... the create method is the one in the Chrome API
17:40:31 <xfq> ... we just make it access multiple models
17:40:43 <xfq> [chunhui shows sample usage]
17:41:21 <martin> ack kenji
17:41:22 <xfq> chunhui: this API uses Chroma DB
17:41:41 <zolkis> q?
17:42:06 <xfq> kenji_baheux: you showed gemini-1.5-flash
17:42:18 <xfq> ... but you also have like an API key
17:42:23 <xfq> ... people would use this app
17:42:31 <xfq> ... I'm really curious about how it's gonna work
17:43:03 <xfq> ... gemini_api_key
17:43:07 <tomayac> q- as my questions was covered by kenji_baheux
17:43:18 <tomayac> q-
17:43:41 <xfq> ... @@ connects to an on-device model
17:43:52 <xfq> ... how it's going to work?
17:44:50 <xfq> kenji_baheux: if the code runs on the browser, then you expose the API key
17:45:11 <xfq> chunhui: yes, the API key will be exposed
17:45:18 <xfq> ... if it's a cloud-based model
17:45:25 <xfq> ... by the browser
17:45:35 <tomayac> Also, would UAs be expected to keep track of API key usage, so the user could find an app that has used up all the tokens…
17:45:46 <martin> ack Dome
17:46:07 <xfq> Domenic: Google is not interested in creating a cross site database to store the info since it breaks the web security model
17:46:32 <xfq> ... if you want to share data between sites you should explore the traditional ways of doing that
17:46:41 <xfq> ... for example using things like authentication
17:46:54 <martin> ack Mc
17:47:08 <xfq> ... should not use some sort of database since it enables cross site tracking
17:47:37 <xfq> McCool: it clearly violates cross sharing restrictions
17:47:46 <martin> q?
17:47:49 <xfq> ... we want each app has its own storage
17:48:01 <cyns> cyns has joined #web-hybrid-ai
17:48:18 <xfq> ... regarding the API key we could generate and API key per session
17:48:29 <xfq> ... limit to the particular browser instance
17:49:00 <xfq> ... where does the model run not enough to identify @@
17:50:23 <xfq> McCool: how do you deal with the web storage model?
17:50:37 <xfq> chunhui: I think authentication is a very traditional topic
17:51:08 <xfq> McCool: different sites have complete different developers
17:52:13 <martin> q?
17:52:21 <xfq> chunhui: let's say one vendor develops 2 different apps
17:52:24 <kenji_baheux> q+
17:52:40 <xfq> ... they have the same user information
17:52:41 <denkeni> denkeni has joined #web-hybrid-ai
17:52:46 <xfq> ... so they can trust each other
17:53:20 <martin> ask kenj
17:53:27 <Roy> Roy has joined #web-hybrid-ai
17:53:27 <martin> ack kenj
17:53:35 <xfq> kenji_baheux: afaiu Huawei has a miniapp ecosystem
17:53:57 <Roy> RRSAgent, make minutes
17:53:59 <RRSAgent> I have made the request to generate https://www.w3.org/2024/09/25-web-hybrid-ai-minutes.html Roy
17:54:05 <xfq> ... it is very relevant to that system?
17:54:15 <xfq> chunhui: maybe miniapp is one scenario
17:54:21 <xfq> ... that can benefit from this API
17:54:50 <xfq> ... but we're trying to say if the API is provided by the native OS whether it is a native app web app or miniapp
17:55:03 <Domenic> q?
17:55:08 <Domenic> q+
17:55:16 <xfq> ... they can share the local device
17:55:22 <xfq> ... the same one user
17:55:28 <martin> ack Domenic
17:56:02 <xfq> Domenic: i think the issue is that the security model for the web is very different from the security model for native apps or mini apps
17:56:16 <martin> q?
17:56:16 <zolkis> q+
17:57:02 <xfq> ... we cannot have the same security model
17:57:14 <reillyg> q+
17:57:19 <xfq> ... i think the cross site storage thing doesn't work
17:57:25 <martin> ack zol
17:57:28 <xfq> ... but the use case is important
17:58:22 <xfq> zolkis: I wanted to ask if you considered @@ then you don't violate the browser security model
17:59:11 <xfq> ... on the edge side
17:59:24 <martin> q?
17:59:46 <kenji_baheux> Domenic: on the other hand, the other part of the proposal (e.g. model switching) is worth exploring further (paraphrasing).
17:59:56 <xfq> chunhui: yes you can implement this on the edge
18:00:02 <McCool> q+
18:00:09 <xfq> zolkis: OK
18:00:14 <McCool> q-
18:00:18 <martin> ack reillyg
18:00:36 <xfq> reillyg: mechanism for loading multiple models, local and remote
18:00:49 <xfq> ... the local model concept is targeting OS built in models
18:00:59 <xfq> ... Gemini running onthe device for example
18:01:32 <xfq> ... what about loading model proveide bythe app
18:01:39 <xfq> ... like webnn and webgpu
18:01:50 <xfq> chunhui: yes
18:01:57 <xfq> ... you can use webgpuo r webnn to integrate a lot of models
18:02:03 <xfq> ... Gemini or others
18:03:12 <xfq> reillyg: i'd be interested in understanding how taht works
18:03:25 <xfq> martin: the security model is a challenge
18:03:35 <xfq> ... this is something we need to solve
18:03:40 <xfq> ... is there any interest beyond?
18:03:47 <xfq> ... if we solve the problem of security
18:04:06 <xfq> McCool: let's follow up in the web machine learning CG
18:04:20 <kenji_baheux> McCool: model switching and RAG aspects (iirc).
18:04:50 <martin> rrsagent, draft minutes
18:04:51 <RRSAgent> I have made the request to generate https://www.w3.org/2024/09/25-web-hybrid-ai-minutes.html martin
18:07:12 <martin> martin has joined #web-hybrid-ai
18:15:05 <martin> martin has joined #web-hybrid-ai
18:15:38 <ktoumura> ktoumura has left #web-hybrid-ai
18:16:57 <Roy> Roy has joined #web-hybrid-ai
18:18:41 <xiaoqian> xiaoqian has joined #web-hybrid-ai
18:19:24 <reillyg> reillyg has left #web-hybrid-ai
18:19:41 <martin_> martin_ has joined #web-hybrid-ai
18:23:20 <xfq> xfq has left #web-hybrid-ai
18:56:50 <Roy> Roy has joined #web-hybrid-ai
19:07:40 <martin> martin has joined #web-hybrid-ai
19:14:50 <Roy> Roy has joined #web-hybrid-ai
19:41:02 <martin> martin has joined #web-hybrid-ai
20:08:12 <martin> martin has joined #web-hybrid-ai
20:08:23 <xiaoqian> xiaoqian has joined #web-hybrid-ai
20:13:55 <martin> martin has joined #web-hybrid-ai
20:16:32 <Roy> Roy has joined #web-hybrid-ai
20:17:42 <kenji_baheux> kenji_baheux has joined #web-hybrid-ai
20:17:56 <kenji_baheux> present+
20:27:44 <martin> martin has joined #web-hybrid-ai
20:34:53 <Roy> Roy has joined #web-hybrid-ai
20:37:06 <kenji_baheux> kenji_baheux has left #web-hybrid-ai
20:52:28 <Roy> Roy has joined #web-hybrid-ai
21:15:40 <martin> martin has joined #web-hybrid-ai
21:16:25 <Roy> Roy has joined #web-hybrid-ai
21:24:46 <martin> martin has joined #web-hybrid-ai
21:29:33 <martin> martin has joined #web-hybrid-ai
21:45:17 <martin> martin has joined #web-hybrid-ai
21:47:06 <martin> martin has joined #web-hybrid-ai
21:48:14 <Roy> Roy has joined #web-hybrid-ai
22:11:16 <Roy> Roy has joined #web-hybrid-ai
22:47:49 <martin> martin has joined #web-hybrid-ai
22:52:11 <Roy> Roy has joined #web-hybrid-ai
22:58:18 <martin> martin has joined #web-hybrid-ai
23:45:37 <martin> martin has joined #web-hybrid-ai