Prepared for the MiniApp Virtual Meeting 2020, July 30-31
Video hosted by Web Castor on their StreamFizz platform.
Hello everyone, welcome to my MiniApp URI presentation My name is Zhou Dan, I'm a member of MiniApp CG and works for Baidu.
As you can see in my agenda I'll start with a case study then move on to the background and syntax of the URI project and lastly our vision for MiniApp URI.
You may realize from the cases we presented previously many miniapps of different functions here are accessible at the same time in one host environment. The host environment here is a platform that allow miniapps to perform on it: it can be either an operating system or a native app. Now the question is how can the host decides which page of which app is the right one to open？
In this case the host environment is a Chinese native app. Its search engine can search information on the Internet and allow the users to open miniapps by simply one click in this native app if there's any miniapp in the SERP. Unlike regular web pages all pages from the same miniapp are of the same format are of the same format aiming for a native-app-like user experience aiming for a native-app-like user experience.
For example, the link of this red area here means opening an index page of a miniapp whose ID is 123. As for the link of the blue area it means opening the animal section of miniapp 123 and sending a query for information about koala at the same time So, what did miniapps like this went through before reaching the users？
Our story begins with the developing process. This process can be broken down into four major steps.
To be more specific, before the developing process start developers are usually required to register on the host's developer management platform and get a unique ID for their miniapp after that With this authorized ID to develop miniapps
During this process, the developers would use a desktop application called MiniApp IDE. This IDE provides a stimulator that shows a real-time preview of the miniapp pages based on the provided source code. It's editable as well
Developers can also preview their miniapp at anytime using the debugging tools in the host's mobile app
Here, their source code would be packaged then uploaded to a temp cloud server and generates a debugged version of URI. This version of URI has a limited valid time. It looks very similar to the published version of URI just with a debug label.
After finishing development, developers would publish their miniapps that process we namely going online. This is the time when their source code be packaged and uploaded to the package server of the host's platform then generates an online URI. This URI would be distributed to all kinds of front ends in different forms.
For example, this screen shot on the left shows a service center that assembled different kinds of miniapps. The one on the top right corner is the smart car system of car networking environment We also have this physical poster with a QR code. These are just some examples of how we distribute the URI.
So, what happened when we access the URI of a miniapp？
Let's take a look at this flow chart. Every time when a user access the URI of a miniapp the host would take two concurrent actions: one is to use HTTPS and request to download the mini app package, the other is to load and initialize runtime environment.
This runtime environment includes a JS logic engine consist of v8 or JS-core and a rendered view container displayed by NA or Webview.
After downloaded the miniapp package JS logic engine would read minifest files namely App.json files then load the JS files of this miniapp, create a page stack and prepare page data. At the same time, the view container would load miniapp template then use the data from logic engine to render pages. After all these steps finished, the user would be able to see the accomplished page.
There's one crucial step in this process also one missing step in HTTP standards marking the resource type in URI. In this scene, taking this step can pre-judge the miniapp and preload the runtime environment. This is the main reason why miniapps run smoothly and fast.
In order to tell miniapps apart from regular HTTP requests every user agent has their respective solutions for identifying. As you can see, there are many hosts in this chart every host clearly has a unique header for their solution.
You may notice that two of Quick App‘s solution look quite similar to HTTP protocol/ Since Quick-App only operates in Android system, Android would in fact identify the character string ‘http://hapjs.org/’ before submiting the protocol to the browser. It will startup Quick App straightaway for this solution instead of going through the browser. Essentially it's very different from the regular HTTP networking protocol.
Let's get back to this chart - I used different colours for different kinds of key information. The blue part shows this solution is for a miniapp. The orange part is the unique AppID for this miniapp in this host environment: each App ID correspond to only one miniapp. The green part is the path for starting this miniapp page up And the yellow part is the query information of this miniapp page.
Though these solutions look quite different at first their key information are all the same. This is why we want to come up with a URI scheme and why it's feasible.
Due to these syntax differences miniapps are restricted in certain host environment: A URI of one platform can only be parsed by this one host that is to say, miniapps are centralize around these hosts. It goes against the principle of one world one web.
Further more, when we look at the architecture of miniapps they usually come as a whole package with all sources inside in order to achieve a native-app-like, smooth user experience.
But for users what they actually need are only some pages in a certain miniapp. So the URI we create must fit the architecture of miniapps and be able to locate certain resource in miniapps as well.
This is the reason why we need MiniApp URI: our goal is to define a general access protocol that can decentralize miniapps. No matter which platform the miniapps originally deployed we hope we could load and parse it on every host platform through one agreed protocol and present it to every user.
We're aiming for navigation between miniapps navigation between web page and miniapp and navigation between OS UI, native UI and miniapp. Our URI will also identify informations about the miniapp that related to accessment including the ID, version, package address and the path to the miniapp page we're going to access as well as query and fragment. This way, every host can identify, parse and locate any miniapp from any platform via this URI.
Here I'm going to show you the designing process of our URI syntax so that you can better understand our designing idea.
At first, in phase one, our initial thought is is it possible to use the package downloading address as the URI? It seems like the easiest way at that time.
However, the answer is "no". Like I mentioned before the host has to pre-judge whether this request is from a miniapp or not and preload the runtime for miniapps. This is what unique about miniapp architecture. So we decided to use the header to do that job. Some user agents told us they want more flexibility they asked is it possible to eliminate the restriction on the way of getting packages.
That brought us to phase two: Later on, we decided that we can meet their request. The name of the domain can be omitted the only required content is the ID of the miniapp for identifying. It provided more flexibility for user agents. The simpliest example of this protocol is miniapp://123 .
Then in phase three we had further discussion: What if the user agent wants to specify the domain name? Surely it's also doable. The domain name is still a part of the authority of our URI just like regular URI protocol. Here's another example just use AppID to replace username.
We have a FAQ for all the discussion about our URI syntax: you may want to take a look afterwards for more information.
Then in phase four we had a new question: Everytime when the miniapp publish it has a version number. Can we use that number to get a certain version of package？
The answer is yes. Though we recommend our users to always get the latest miniapp package to get full function like the web pages; there may still be some developers who want a specific version of miniapp for compatibility reasons.
So we set a version field: While an app ID can unique identify a specific miniapp an AppID and a version number together an AppID and a version number together can unique identify a certain miniapp package.
The significance of version number is the same as the app ID in our URI so we used semicolon here for segmentation and placed version number behind AppID. Here's an example.
In the following phase five we discussed where we should place the path field: Let's take a look back and try to remember what did other user agents do.
Some hosts put path as a query in their URI. But after some discussion, we decided that it would be better if we have a field for the path of miniapps it makes more sense. Besides, it will make a good separation from the query of miniapp page query. This is an example with path field.
Then, where should we place the query of a certain miniapp page? Like I mentioned before all key information, like app ID, version number and path have alredy settled in the most suitable place. The query of miniapp page is not different from the query of regular URI and will be placed in the query field in URI. The same goes for fragment.
You can see a final example on the bottom with all fields.
Next part, I'm going to introduce the syntax design of our URI In the whole protocol scheme is a fixed string, miniapp used for pre-judging miniapps The authority part includes key information for miniapp location. It's consist of ID, version number and host and ports for package obtaining. The ID can be of any length, and it's a unique string under a specific host it's usually assigned by the host. Together with a version number, it points to a unique miniapp package.
Host and port field is optional. When host and port are null the user agent that opens this URI gets to decide how to open the miniapp with specified ID as the default behavior. When host has a value the user agent can obtain the miniapp package via network protocol or file protocol and open the miniapp page. For now, before miniapps get decentralized the host usually orient to the domain of the miniapp producer.
Let's take a look at this complete example: every part of it correspond to field of our URI. If you want to get more information about our syntax design you may check our miniapp URI scheme proposal.
Now I'm going to show you some key scenarios of miniapp URI.
Everytime a user access a URL the host would check the scheme to see whether it's a miniapp. If it's a HTTPS protocol then it's just a normal web request the host would send the request to the server. But if the host detects a miniapp header then it would know this is a miniapp and use the agreed protocol to parse the whole URI.
Following some rules it can then get the package with the parsed information. Like I mentioned before the host may get the package in local or through HTTP request or even through other protocols. There are many ways available.
After getting the miniapp package host would open the miniapp then open some pages based on what it got from the miniapp protocol.
In this example, the host opened pagedetail page About our miniapp URI there's one issue that we discussed a lot.
Can we simply use HTTP url as miniapp URI? Let us assume that if we use HTTP as miniapp URI since we don't actually have a centralized server the domain, like this miniapp.com here might be our only choice as an identifier for miniapps to be recognized. It's the pre-judge step we discussed before.
Now, let's take a closer look at this flow chart. If we use HTTP protocol then our URI would looks like the long string on the left. It seems not quite conform to HTTP semantic.
When a user agent access a HTTP URL like this it would check the scheme first. If the scheme is HTTPS then the user agent would intercept it and judge wether the first part of this string is https://miniapp.com/ If so, then this URL is a miniapp. It's a bit like Quick App's approach we mentioned before.
After that, the user agent would parse the URL and get some key information like its host, ID and version number and the path for opening it as well as the query and hash of this path.
Next, the user agent would apply some rules to get the package. The ways of getting package are no different from our precious flow A. It's still up to the user agent. It can either find the package in local or HTTPS request it or user other protocols to download the package.
Then finally, open the miniapp page. The advantage of using HTTP URL is that we don't need to create a new scheme we can just use the existed HTTP scheme.
But the disadvantages here are also obvious.
There's yet another possibility! If we can have a global centralized service everything may get a bit easier. Here's an example of a HTTP link on the left. It might be an address for getting a package When a user agent access such a URL again, it would check the scheme first. If it is HTTP scheme then the user agent would send a request to the domain miniapp.com. And then then the server of miniapp.com will judge the UA to see if it support this miniapp.
If the answer is yes then the server would use the information in this URI scheme like version number, path and query to send back a miniapp package as response.
After getting this package, the user agent would then be able to open this miniapp via assigned path On the other hand, if the server checked the UA and decided that it doesn't support this miniapp it might send back a 400 status code or redirect to a page that can provide similar services to this miniapp.
This kind of flow has a clear advantage. We don't need to create a new URI scheme or a new URI syntax. But its disadvantages are also significant. Like I said several times in this presentation miniapps need pre-judgement. This process here doesn't have a pre-judge step thus is lack of anticipation for preparing runtime environment.
Surly we can still strong-match the string miniapp.com like we did in plan B Then like plan B, there would till be the problem of redundant. The second disadvantage of this plan is that it relies on a centralized service but such centralized service doesn't exist for miniapps yet. Besides, this plan also has user agents restricted. The only way of getting miniapp package here is via HTTP request which is not flexible enough and can't meet the need of some user agents.
We are still discovering more possibilities for our miniapp URI you are welcome to our issue page to tell us your insight.
For long-term vision we hope miniapp could go global and be accepted by more browsers and app developers. We also hope that miniapps can run across platforms, hosts and browsers without any difference.
And ultimately, we hope that miniapps can have a worldwide centralized management system and miniapp://id can be an unique identifier for all miniapps across the world. These are our visions for miniapp URI in the long run. That is my presentation about miniapp URI, thank you for listening Your comments and suggestions are mostly appreciated. Thank you