MiniApp Standardization White Paper

1.4.1 Case 1: Shared Bicycle Service

The popularization of MiniApps helps to make the shared bike a seamless service instead of a cumbersome application.

• User chooses any of the miniapp platforms on their mobile device, which, is usually a super-app that he/she already logined;
• User scans the QR-code label attached on a shared bicycle within the superapp;
• The hosted-app will automatically navigate to the shared bicycle miniapp and unlocks the bicycle instantly;
• Upon arrival, user locks the bike on the miniapp;
• Transaction completes, a message of the payment detail is sent to the user.

For users, miniapps can bring convenient from a few perspectives:

1.4.2 Case 2: AR Zoo

MiniApp developers can simply use HTML/CSS/Javascript as the programming language, but MiniApp is more flexible so it excels at quick development of complex features in daily work.

This MiniApp tries to build an AR zoo with AI technologies to recognise the animals. Developers can easily do that by adding a few components or APIs that provide access to the native capabilities or advance features, f.ex., Image Recognition, AR 3D Animal models rendering, a speech API to for speech synthesis, AR navigation provided by the map SDK.

MiniApps can be discovered by the search engines, by the MiniApp store in the hosted-app or by QR-code.

For the developers, the incentive to work on miniapps is very obvious.

1.4.3 Case 3: MiniApp for IoT

One of the goals of MiniApp to connect information and services across different platforms, so it's ideal for IoT applications such as Smart Automotive, voice-control speaker and Smart TV.

Nowadays, it's possible to convert some MiniApps to adapt the vehicle screen and system, also, a few MiniApp vendors have built a MiniApp platform specially design for the vehicle system, to help distribute or upgrade applications to various of car model. This brings millions of Web developers to the Automotive application ecosystem.

These Automotive miniapps can service many user scenarios, including gas filling, car washing, Electronic Toll Collection, insurance, restaurant reservation, or entertainment. For example, when the system detects the remaining fuel is less than 20%, it can recommend to the owner a Gas Pumping MiniApp. The user can get the information of the nearest gas station and head there to complete the gas pumping including the payment within MiniApp, "refueling without getting off the vehicle".

2.1.1 Separate the View Layer from the Logic Layer

In a miniapp, the view layer is usually separated from the logic layer.

The View Layer is responsible for rendering MiniApp pages, including Web components rendering and native components rendering, which can be considered as hybrid rendering. For example, Web components rendering can be handled by WebView, but some of the Web components rendering aren't supported by WebView or have performance limitation, MiniApp also relies on some native components, such as map, video, etc.

Logic Layer is implemented with JS Worker. The Worker is responsible for MiniApp’s event processing, API calling and lifecycle management.

Extended native capabilities usually come from the hosting Native App or OS, including payment, file processing, scanning, phone call, etc. These capabilities are called through certain APIs. When a Mini App calls a native API, it transfers the API calling to Extended Native Capabilities for further processing via JS Bridge, and obtains the result from Extended Native Capabilities via JS Bridge.

Worker establishes connection for each Render, transferring the need-to-be-rendered data to render for further processing.

If an event is triggered by a component in a MiniApp page, this page’s Render will send the event to worker for further processing. At the same time, the Render will wait for the data sent from worker to re-render the MiniApp page.

The rendering can be considered as stateless, and all the states will be stored in the worker.

The benefits of separating the View Layer and the Logic Layer include:

• Very handy for data sharing and interaction among multiple MiniApp pages.
• Having the same context within a lifecycle of MiniApp can provide a similar coding experience for those developers coming from native app developement background.
• Separation and parallel implementation of Render and JS worker can prevent the situation where a JS execution impacts or slows down the page rendering, which can help enhance the performance of rendering.

2.1.2 Rich APIs and Components

MiniApp platform provides many components to help developers build a fancy UI, including basic components like View, Form, Image, and high-level component like Maps.

MiniApp venders also offer a number of APIs for developers to get access to capabilities of both the Web and the native, including basic APIs such as UI display APIs, Image API, and those advanced ones like Account API, Map API, Payment API.

APIs usually work together with components. When user clicks certain component on a MiniApp page, it will call the related API to complete user’s interaction and refresh the current MiniApp page if needed.

2.1.3 MiniApp Constructor

In order to obtain similar user experiences to native apps, MiniApp resources are usually packaged together. After downloading and installing a MiniApp package, all static page templates/CSS/JavaScript files need to present app pages already exist on users' machines. These resources are always available without any redundant downloads until next update.

A MiniApp package is a compressed format (e.g. zip) archive, including:

• one configuration document, located at the root directory of the package. The config file should include:
  • general description of the entire MiniApp.
  • description of pages and their corresponding paths, for the purpose of page locating and opening.
• one app-level logic file, containing JavaScript codes, deals with app-level lifecycle callbacks.
• one or multi page files, containing templates codes for page structure, CSS codes for page style, and JavaScript codes for page logic.
• support digital signature for validation.

For the purpose of locating a specific MiniApp while searching and executing, a MiniApp must have a package name or an ID on the platform. An icon is also required for user recognization.

2.1.4 MiniApp Widgets

In addition to being presented in the form of an MiniApp page, MiniApp can also be displayed in the form of information fragment, that is, a MiniApp widget. In this form, developers can put their service and/or content to various host scenarios, called host environment, such as assistants, search page, etc.. This feature connects services of the MiniApp with the scenario, providing users with more convenience.

For example, when a user purchases a train ticket for trip, the MiniApp widget on smart assistant shows the train's latest status immediately. The user can click on this widget and jump to a full-screen MiniApp page for more detailed information.

Same as MiniApp page, widget is also described by URI scheme. The host environment specifies the MiniApp package and the corresponding widget to be loaded through URI path, and passes data to the widget through URI query params. After the widget is loaded, it is displayed and rendered in the host environment. Data from the host and widget, as well as data from different widgets are isolated to ensure security and independence.

In many scenarios, a widget can open a MiniApp page for more complicated operations. In such cases, widgets often need to share data with its corresponding MiniApp, e.g. maintain a consistent login status. Therefore, the data of the widget and the MiniApp can be accessed from each side. In other word, MiniApp widget and page have same data access rights.

One of the goals of the widget is to let the user forget the traditional app concept and truly meet the user's needs in the form of service. So in addition to all app invocation paths, widgets can also be triggered by different methods on different scenarios, such as text keywords, voice analysis, picture recognition, scan code, event intent.

2.1.5 Single-Instance, Multi-Entries

There are multiple entrances to discover, open and access MiniApps. Unlike Web in multi WebView, only one instance will be created for the same MiniApp, so MiniApp keeps its status and data in a consistent manner globally. For example, after one user opens and logs in a MiniApp through the entrance of QR code for the first time, the user will remain logged in next time when the user returns from another entrance like MiniApp store.

The entries for MiniApps include, but not limit to:

• MiniApp store
• Search Engine
• Smart Assistant
• QR code
• SMS/text
• Physical Object(with AI)
• Browser
• Calender items
• Voice message(with AI)

2.1.6 Performance and User Experience

MiniApps try to improve their performance and user experience with a few mechanisms that have been proved to be effective through practice.

Packaging

With MiniApp's constructor, user only needs to download the package when the MiniApp is first opened, and then the static resources(pages/JavaScript/CSS) in the MiniApp needn't be downloaded again, so that the loading and jumping of the following pages can be more quickly responded. This feature improves user operation experience and save network traffic.

Meanwhile, the MiniApp has a pre-download mechanism, which can download the MiniApp packages in advance, or pre-downloads separately for the first page, and perform streaming decompression in parallel during download to minimize the time-consuming of MiniApp startup phase and balance the loss of the first page performance when openning first time.

Multiple Rendering Views

The MiniApp uses native page stack management between render views, and the page switching is driving by native code. Therefore, the gesture operation in the page, the switching between pages can achieve the exactly same smooth experience as Native.

Becase of the isolation of View layer and Logic layer, the View layer can be rendered independently. Without being blocked by JavaScript logic code, the rendering speed of the page can be greatly improved.

Pre-built and reuse of runtime environments

MiniApp's runtime environment is usually pre-built before launching a mini-app, thus reducing the time to launch a MiniApp. Pre-built content includes rendering views, static resources, developer defined prefetch requests, and MiniApps runtime container. After a MiniApp is activated, it will take over the pre-built rendering view, then we continue to pre-built a new render view into the cache pool for the next one. There is a limit on render view quantity, when any render view gets closed or the quantity limit is exceeded, the oldest opened render view will be destroyed. When the MiniApp application exit, the runtime will be destroyed and the application environment and resources can be reused.

Pre-defined Component and API

MiniApp platforms provides very rich components and APIs, these components and APIs are usually well designed and able to help ensure the performance of the general work from developers.

JavaScript framework presets and hot reload

MiniApp's runtime environment contains two major parts, the basic capabilities provided by Native code, and a framework, include developer API and some components, implemented by JavaScript. The JavaScript framework is built in native Apps, and will be loaded into the MiniApp runtime environment in advance before executing MiniApp. The JavaScript framework can be hot reloaded (reload during usage), which brings lots of possibilities to improve performance.

3.1.1 Hybrid Rendering

MiniApp is a hybrid solution of Web rendering and Native rendering. It'd be great if there is a good way to combine the rendering results from the Web and from the Native.

Proposal: Mini App needs a standardized API to help integrate the native rendering result into Web rendering result.

3.1.2 Transition Animation

MiniApp would like to provide transition animation during page switching so that users can have similar experience as when they are using a Native App, but it's almost impossible to do that now.

Proposal: Mini App needs an API to call add transition animation during MiniApp page switching.

3.1.3 Standardise the Package Constructor of MiniApp

MiniApp can form a package and parsing convention for multiple MiniApp hosting platforms through a standardized distribution format. Currently, each MiniApp hosting platform provides different development tools (different packaging methods), and the MiniApp is parsed differently in different MiniApp hosting environments.

Proposal: The MiniApp is actually a packaged (compressed) collection of files during the distribution process. We can describe a MiniApp (.ma) with a uniform file suffix and specify how to create the .ma file and how to parse the .ma file.

3.1.4 Standardise the navigation to a MiniApp page

For a hot page in a MiniApp, it may be referenced in another MiniApp, and it is expected to be accurately evoked when the user visits.

Proposal: Define a standardized protocol (URI scheme) to access MiniApp.

3.1.5 MiniApp Widgets

Like Android widgets or apple dashboard, users can directly get information and/or finish their task by a MiniApp widget without opening any web or app pages. A MiniApp widget can be displayed in an environment outside of web browsers, such as desktop or dashboard.

Proposal:

• A MiniApp widget can be displayed within a host environment, which could be either webview or a native app page. The host environment loads a widget with its corresponding URI path, which describes a package and widget page.
• A MiniApp widget can access local data or data from server. Meanwhile, a MiniApp widget can communicate with MiniApp in the same package.
• A MiniApp widget should be interactive, meaning that it should be responsive to any user behaviors/interactions. A MiniApp widget should have the ability to open a web or app page.

3.2.1 Define an event of time to interactive in MiniApp

MiniApp needs to know when the MiniApp page Time to Interactive (TTI) has been completed.

Proposal: A standardized event to notify that the MiniApp page Time to Interactive has been completed.

3.3.1 3D Model Element

3D models become more and more popular thanks to its rich details, and combined with AR, it will provide a much better user experience than 2D. The business cases might include online shopping, advertisement, education, etc. However, the current web lacks a standard and convenient way to deal with 3D model. In this document, we propose to define a HTML tag to directly handle 3D model, similar to the way we handle audio, video, image with corresponding HTML tags:

• 360 view

  User can view a 3D model from different angles via gestures. And the 3D model can be zoom in/out as well. It can be viewed in full screen, or it can be embedded in a HTML page, shown together w/ other HTML contents.

• View with AR

  User can place the 3D model in the real world environment with camera. User can specify different locations to place the model.

Proposal: A <xmodel> Element to specify a 3D model on the web and to power interactive 3D contents with AR.

3.3.2 Face Tracking

Face tracking can be used in many 3D scenarios.

Face Effects in Live Video

Add effects on the face in live video. These effects include full-screen filter, face reshape and makeup, 2d sticker, 3d headdress, etc. Most of these effects depend heavily on the real-time face tracking from video source.

Gaming

Game developers can design game strategies based on the tracked face. Like triggering a specific game logic when the eye is blinked, or check if the dropping fruit is in the opened mouth.

Virtual Makeup

Let users try lipsticks, eye shadows, glasses, hats on the product page to help them make the decision.

Proposal: A Face Tracking API use a video element as input and updates face tracking output every frame, which includes:

• Bounding box of each face
• 4x4 pose matrix of each face
• Normalized (x, y) landmarks points
• ace geometry data including vertices, normals, texture coordinates

3.3.3 Hand gestures tracking and recognition

Hand gestures can be used in video effect and AR/VR games scenario which can make apps more impressive and interactive.

Proposal: A high-level API to track hands movement, get hands outline.

3.3.4 Low level AR APIs based on ARCore and ARKit

There are a few AR APIs in MiniApps we would migrate to the Web, as they help to provide better AR experience in gaming, 3d model preview, interactive ads.

Proposal: Provide low-level AR APIs based on ARCore and ARKit, which includes:

Camera view matrix for world tracking

Provide the 4x4 view matrix of the spatial location and orientation of the mobile phone, which can be used to update the camera matrix in their 3D virtual world in real time by developers. Thereby the position of the real world can be correlated with the position of the object in the virtual world.

Plane detection and tracking

Detect planes in the real world and track these planes in real-time. Provide the 4x4 transform matrix which represents the center position and orientation of each plane. It can be used for placing 3D virtual object on the ground/desktop.

Anchor

Anchor defines a fixed position and orientation in the real world. Developers can create an anchor from a 4x4 transform matrix, which can be got by hit test. This matrix will be updated each frame to ensure that the virtual object corresponding to the matrix can be fixed in one position and direction in the real scene.

Hit test

Get a 4x4 transform matrix representing a position and orientation in the real world space corresponding to the screen position to implement functions such as clicking and placing virtual objects.