Client-Edge-Cloud coordination Use Cases and Requirements

A Cloud App is a new form of application which utilizes cloud computing technology to move a client side application's workload to the Edge Cloud and/or a Central Cloud. The user interaction happens at the client device side and the computing and rendering process happens in the Edge Cloud side. This can accelerate the client side application's performance, lower the client's hardware requirements, and reduce cost.

As one example of a cloud App, Alibaba Group's Tmall Genie smart speaker leverages the Edge Cloud to offload its computing intensive and accelerated workload from the client side to the Edge Cloud.

The client, the Central Cloud, and the Edge Cloud work together in a coordinated way for Cloud Apps. Typically, the control and orchestration function is located in the Central Cloud. The computing intensive function is located in the Edge Cloud. The user interaction and display function is located in the client.

VR/AR devices such as VR/AR glasses normally have limited hardware resources, so it is preferred to offload computing intensive tasks to Edge Resources for acceleration. This reduces delay since the Edge Resource is deployed near the location of the user.

Note: this could be generalized to "acceleration of low-latency tasks". Some other examples might include game physics simulation or CAD tools (in a business environment). The latter might add confidentiality constraints (a business user may want to offload to on-premises computers). This pattern is for local communication to/from the client. See also "Streaming Acceleration", where the communication is in-line with an existing network connection.

Cloud gaming is a game mode which leverages Cloud and/or Edge Resources. Under the operation mode of a cloud game, the games are running on the Cloud side, and the game images are compressed and transmitted to users by video stream through the network after rendering. The cloud gaming user can receive the game video stream and can send control commands to the cloud to control the game.

Taking a "Click-and-Play" scenario for example, since all the rendering and control commands are offloaded to an Edge Resource, the cloud game user doesn't need to install the game locally. The user can just click the game and then play it easily.

By offloading the gaming workload to the edge and making full use of the computing power and low-latency network, the cloud gaming can provide a smoother experience.

In the case of video acceleration, we may want to offload work to a location with better compute performance that is already on the network path. Specifically, consider a low-performance client that wants to compress video or do background removal as part of a teleconference. It could connect over a local high-performance wireless network to a local edge computer (perhaps an enhanced ISP gateway box) that would then perform the video processing and forward the processed video to the network.

Note: this pattern is for communication "in-line" with an existing network connection. See also "VR/AR", where the communication is to/from the offload target.

Machine learning inference can be done in the client side to reduce latency. W3C is working on the WebNN standard that allows client side developers to leverage the machine learning acceleration hardware that resides in client side devices.

The client side devices may have different hardware capabilities. For example, some mobile phones have very limited hardware resources and do not have a GPU. It is very difficult for this kind of client device to run machine learning code. So in this scenario, it is preferred to offload machine learning code to the Edge Cloud. The user experience is greatly improved in this case.

4.1.5.2 Professional Web-based Media Production (UC-PWMP)

Another special case and example of machine learning acceleration is professional web-based media production. Processing and rendering media is a complex task. For example, a video editing application needs to do image processing, video editing, audio editing, etc., so it has high performance requirements.

Professional web-based media production heavily relies on web-based media editing tools, and may include AI-based operators which can be used to do cutting, editing, transcoding, and publishing of videos to the cloud. Since the Edge Cloud has more computing power and is close to the user's location, by offloading the expensive rendering process to the Edge, web appliations can render media more quickly and provide a better user experience.

In general, there are many opportunities for IoT devices to offload work to another computer, since they are generally constrained in processing power, energy, or both. Video processing however is of special interest because of its high data and processing requirements and privacy constraints. In this case the offload source is not necessarily a browser or web application but the process may be indirectly managed by one, and the capabilities provided by the proposed standard (e.g., a discoverable compute utility) can also be used directly by an IoT device.

Currently, there are two common approaches for offloading. One is to send the code in the request from the client to the Edge Resource, the other is to have the Edge Resource fetch the code from file repositories on the Edge and execute them. Both work fine, but both approaches also have some downsides.

Sending code from client to edge

In this approach, the client sends the local code to the Edge Resource, which then executes the code on the Edge side. The downside is obvious: since more data is transferred in the request it may cause network latency. Handling more data will also put more strain on the resource-constrained end device. Meanwhile, some code is sensitive, data security is also an big issue.

Fetching code from file repositories and executing on the edge

In this approach, the client leverages a user-defined offloading library to send proper parameters to the Edge Resource. The Edge Resource will then fetch the specified code from repositories and then execute the code.

The downside of this approach is that additional file repositories are needed and the developers have to upload the code to the repository and make sure the local and the Edge have the same version of the code.

Meanwhile, since the offloading library plays an important role in offloading, developers need to create a robust offloading policy to discover and connect with Edge Resources, decide which parts of the code can be offloaded, and the time to offload. This will put more strain on the developer and affect the overall programming experience and the productivity.

The two approaches discussed above are mostly similar. The main difference is the way code is sent to the edge. However, in addition to the downsides mentioned, some common issues or pain points still remain and need to be addressed.

• Discrepancies between the client runtime and the Edge runtime may cause failures, as well as differences in the runtime between one Edge Resource and another. So a unified runtime needs to be considered. A WebAssembly runtime may be a good choice.
• Capabilities for discovering and connecting with Edge Resources are needed.
• Some parts of the code might be sensitive, and the developer might not want it to be sent over the internet. Capabilities for developers to configure which parts can be offloaded are needed.
• When certain conditions are met, such as if there is no Edge Resource in proximity to the user or the internet is lost, code should be executed locally instead of offloaded. That is to say, the developer will need to know what might be offloaded to a server but does not need to decide or care about when it is offloaded.
• Security and privacy are important, so secure communication mechanisms are needed.

There is no W3C standard currently available to address the above gaps. To achieve interoperability between different vendors, standardization is needed.

The following are a set of high-level requirements, cross-referenced with related use cases.

Some more detailed requirements are listed below, cross-referenced with related use cases and related high-level requirements.

This architecture allows the client, Edge, and the Central Cloud to share a common code running environment which allows the task running in either client, edge, cloud or both in a coordinated way.

The proposed high level architectures is shown in the following figure:

In this architecture, the code of the workload in the client side could be offloaded to the Edge Cloud and can also be handed to the Central Cloud and handed back to client. The high-level procedure is as follows:

• The client side application encapsulates offloaded workload code into an offloaded code module and loads the code module using specific APIs for the runtime environment. There may be different types of runtime environment, for example, WebAssembly or JavaScript runtimes. The offloaded workload will be written according to the specific runtime.
• The client's runtime dispatch policy module queries the dispatch policy from the Offload Management Module which is located in the Edge Cloud or the Central Cloud.
• The Offload Management Module sends a dispatch policy to the client side application.
• The client side runtime sends the offloaded code to the target Edge Cloud/Central Cloud according to the dispatch policy.
• The target Edge Cloud/Central Cloud executes the offloaded workload and returns the result to the client application.
• If offloading is not feasible in certain conditions, for example, if the network connection between the client and Edge Cloud is not stable or there are not enough resources in the Edge Cloud, the offloaded workload should handover back to the client side to ensure the availability of the client application.

The Web already has a set of standards for managing additional threads of computation managed by application code loaded into the browser: Web and Service Workers. Workers already support a message-passing style of communication which would allow them to execute remotely. This architectural option proposes extending Workers to support edge computing as follows:

• Compute Utility services would be made available on the network that could provide a capability to execute Worker payloads. For fallback purposes, the client itself would also offer a Compute Utility to support local execution when needed. It would also be possible for the origin (the server) to host a Compute Utility. However, in general Compute Utilities could be hosted at other locations, including in desktops on the LAN, within a local cloud, or within edge infrastructure.
• Application developers would continue to use existing APIs for Workers, but could optionally provide metadata about performance and memory requirements which could be used to select an appropriate execution target.
• Browsers would collect metadata about available Compute Utility services, including latency and performance, and would select an appropriate target for each Worker. The user would have controls in the browser to control this rule, including the ability to specify specific offload targets or to force local execution when appropriate. Note: the reason it is suggested that the browser makes the decision and not the application is to prevent fingerprinting and associated privacy risks. Metadata about available Compute Utilities might otherwise be used to infer location. The proposed architecture hides this information by default from the application while still supporting intelligent selection of offload targets.
• Once a Compute Utility is selected, the browser would automatically (transparent to the application) use the network API of the Compute Utility to load and execute the workload for the Worker. Note that in the existing Worker API, a URL of a JavaScript workload is provided to the Worker.
• The Compute Utility itself would be responsible for downloading the workload, it would not have to be downloaded to the browser and then uploaded. Also, a WASM workload could be used, bootstrapping from the standard JavaScript workload. In general, the workload execution environment should be the same as the normal Worker execution environment. However, access to accelerators to support performance and other advanced capabilities will become more important in this context.

The proposed high level architecture is shown in the following figure. The browser discovers Compute Utilities using a Discovery mechanism. Discovery services in each discovered Compute Utility return metadata, which then allows the selection of a Compute Utility for each workload. A Workload Management service is then used to load and run a packaged workload for the worker.

This proposal proposes to extend the WebAssembly runtime and use it in both client side and Edge Cloud as a unified runtime.

The proposed solution includes the following parts:

Load WASM code

The client loads the WebAssembly code by invoking a standard API. This API should indicate that the WASM code could be offloaded to Edge cloud when certain conditions are met. This API should also set the destination Edge cloud's location identifier.

Offload WASM code to Edge Cloud

The client side's WebAssembly runtime executes the WASM code and when certain conditions are met, it offloads the WASM code to the Edge cloud.

Run WASM code on Edge cloud

The WASM code runs on the Edge cloud, and return the result back to the client side.

Dispatch back to the client

When certain conditions are met, the WASM code is dispatched back to the client side and continue to run.

JavaScript API to specify workloads:

This JavaScript API is to be used by a web application to specify a workload. It indicates this workload may be offloaded to the edge cloud when certain conditions are met. The standardization of this API is required since different implementations should implement this API in the same way to make sure that application developers have the same user experience.

Communication mechanism and protocol between client and Edge Cloud:

The communication mechanism, the communication protocol, and the interface (network API) between the client side and the Edge cloud should be standardized to enable the interoperability of different Edge Cloud providers.

Edge Cloud availability and network condition discovery:

The workload should only be offloaded when certain conditions are met. These condition may include Edge Cloud availability and network conditions.

The following topics will be out of scope of standardization work:

Offloading policy:

This proposal recommends that the offloading policy is not standardized to allow flexible implementation by different vendors.

As analyzed in the gap analysis section and in the potential standards section, the proposed potential standards work is to specify a standardized common mechanism to offload code to the Edge Cloud. There may be multiple potential solutions and some of them may be related to WebAssembly and some may be related to web platform APIs.

The proposed next step is to continue exploratory work on potential architecture and standardization proposals by forming a W3C Community Group, which would allow collaboration on solution designs.

This Community Group could firstly develop a standard solution architecture. If the standard solution architecture requires extension of current W3C standards, it would work together witn related Working Groups (for example, WebAssembly Working Group, etc.) to develop the extensions.

If new W3C standards are needed, after developing specifications in the Community Group, the Community Group could propose creating a Working Group for standardization.