Information

Web API for Hybrid AI
  • Past
  • Confirmed
  • Breakout Sessions

Meeting

Event details

Date:
– Pacific Daylight Time
Status:
Confirmed
Location:
2 Ballroom Level - California A
Participants:
Denis Ah-Kang, Martin Alvarez-Espinar, Kenji Baheux, Rijubrata Bhaumik, Domenic Denicola, Javier Fernandez, Ningxin Hu, Iris Johnson, Jesse Jurman, Zoltan Kis, Rob Kochman, Toshiaki Koike, Mirja Kühlewind, Evan Liu, Sandor Major, Michael McCool, Chunhui Mo, Elias Selman, Cynthia Shelly, Shunya Shishido, Austin Sullivan, Kunihiko Toumura, Hannah Van Opstal
Big meeting:
TPAC 2024 (Calendar)

Web apps are increasingly expected to gain access to a language model. We are proposing Web APIs that allow web developers to directly access both on-device and cloud-based language models, and securely share user data between multiple apps when using these models.

The following are the APIs goals:

  • Provide web developers with a connection strategy for accessing both on-device and cloud-based models. For example, if no on-device models are available, attempt to access cloud-based models. Conversely, if cloud-based models are unavailable, try accessing on-device models.

  • Provide web developers with a storage strategy for sharing user's private data. For example, one web app saves users' private data into a local vector database. Another web app, when accessing a on-device language model, can leverage this data through a local RAG system.

The following are not within our scope of concern:

  • Design a uniform JavaScript API for accessing browser-provided language models, known as the Prompt API, which is currently being explored by Chrome's built-in AI team.

  • Issues faced by hybrid AI, such as model management, elasticity through hybrid AI, and user experience, as this topic has already been discussed in Hybrid AI Presentations in the WebML IG, and will be covered in the sessions on AI Model Management.

This session will showcase how to use our proposed API for booking flights and hotels. It will also provide specific implementation details and references for these APIs. Example source code and implementation references can be found on GitHub web-hybrid-ai.

Agenda

Chairs:
Chunhui Mo, Martin Alvarez-Espinar

Description:
Web apps are increasingly expected to gain access to a language model. We are proposing Web APIs that allow web developers to directly access both on-device and cloud-based language models, and securely share user data between multiple apps when using these models.

The following are the APIs goals:

  • Provide web developers with a connection strategy for accessing both on-device and cloud-based models. For example, if no on-device models are available, attempt to access cloud-based models. Conversely, if cloud-based models are unavailable, try accessing on-device models.

  • Provide web developers with a storage strategy for sharing user's private data. For example, one web app saves users' private data into a local vector database. Another web app, when accessing a on-device language model, can leverage this data through a local RAG system.

The following are not within our scope of concern:

  • Design a uniform JavaScript API for accessing browser-provided language models, known as the Prompt API, which is currently being explored by Chrome's built-in AI team.

  • Issues faced by hybrid AI, such as model management, elasticity through hybrid AI, and user experience, as this topic has already been discussed in Hybrid AI Presentations in the WebML IG, and will be covered in the sessions on AI Model Management.

This session will showcase how to use our proposed API for booking flights and hotels. It will also provide specific implementation details and references for these APIs. Example source code and implementation references can be found on GitHub web-hybrid-ai.

Goal(s):
Explore the potential of our proposed Web API for accessing hybrid AI through use case demonstrations and API implementations. Additionally, discuss concrete steps for moving forward.

Agenda:

  1. Introduce the goals of the Web APIs we propose for hybrid AI (2m)
  2. Introduce Connection API (5m)
  3. Introduce Storage API (4m)
  4. A Showcase of Hybrid AI App (3m)
  5. Considerations for Connection Strategy (2m)
  6. Considerations for Storage Strategy (2m)
  7. Considerations for Native OS APIs (2m)
  8. Discuss possible resolutions, followup actions and collaborations (20m)

Links

Materials:

Track(s):

  • AI

Minutes

 Read minutes

Export options

Personal Links

Please log in to export this event with all the information you have access to.

Public Links

The following links do not contain any sensitive information and can be shared publicly.

Feedback

Report feedback and issues on GitHub.