Use case for Web API
accessing LLMs data

Hi everyone, my name is Chunhui Mo. I currently work at Huawei. My topic is: Use case for Web APIs accessing LLMs data.

Nowadays, more and more web applications running in browsers offer AI assistants to users.

These AI assistants usually collect user data and send it to backend services via a public network.

Meanwhile, the app's backend requires an AI proxy service to manage multiple LLMs.

We've found some problems with the above case.

The AI assistants need to collaborate to accomplish a comprehensive task.

Sending private data raises user concerns about security and privacy.

The AI proxy increases development costs and reduces performance.

Let's discuss the causes of these problems.

For example, when making travel plans, the AI assistant for hotel booking needs to know the plane's arrival date to determine the hotel check-in time.

The AI assistant chooses a suitable hotel based on the user's preferences. However, the leakage of travel arrangements can raise security concerns.

Different LLMs have different authentication, request parameters formats, and response data formats. That means a proxy service is needed to handle these differences.

If new Web APIs can access LLMs data, we can expect the following:

A trustworthy environment for collaboration between AI assistants.

AI assistants can exchange private data with local customized LLMs.

Direct access to LLM data through unified APIs.

Let's move on to the first point.

Just like the AI Agent, each AI assistant is authenticated and capable of communicating with others.

Collaboration occurs not only among web apps but also among mobile and PC apps.

The sample code looks like this:

We create flight and hotel assistants using secure configurations for both, as well as LLMs. The hotel assistant retrieves the arrival date from the flight assistant, then recommends suitable hotels for the user.

Let's move on to the second point.

AI assistants can collect private data to train local customized LLMs.

Only authorized AI assistants can access the private data from local LLMs.

The sample code looks like this:

This time, we use an embedding model to create the hotel assistant, then convert user preferences into embeddings before adding them to a vector database. The hotel assistant then uses these preferences to recommend suitable hotels.

Let's move on to the last point.

We propose to standardize the request and response formats for accessing LLM data.

The unified APIs will NOT allow private data to be sent to remote LLMs.

The sample code looks like this:

At first, we create a flight assistant. This time, we use unified APIs to access a remote LLM. An error occurs when attempting to transmit the user's preferences to the remote LLM.

Furthermore, I have one question for you all:

How can we standardize the Web APIs accessing LLMs data?

That's all, thank you!