Webizing Virtual and Augmented Reality Content

Authors: Sangchul Ahn (Letsee)

Introduction

With the consecutive introductions of various enabling devices, Virtual Reality (VR) is facing the next wave for becoming popular. As a technical subfield of VR, Augmented Reality (AR) is also preparing to scale out on the basis of considerably stabilized computer-vision technologies. The widespread use of VR/AR premises its eco-system with a scalable contents platform as the center. The platform should support the efficient production and provide wide distribution channel.

Web has been publicly qualified as a scalable contents platform over last two decades. Its design principles [1] consisting of simplicity, modular, and decentralization have made it enable to easily create various contents by reusing existing external resources, simply publish the contents just by putting into a Web server, and dynamically update the contents at runtime even by end-users after distribution. And there is no doubt that these capabilities have arisen from HTML (HyperText Markup Language) and its principle, Web architecture [2].

However, many content specifications for VR/AR which attempt to adopt Web-friendly approaches [3,4] have been experiencing difficulties to achieve a consensus because their independent structures were not compatible with Web architecture and were difficult to take the advantages from Web eco-system. We believe that to support VR/AR as the content platform, Web technologies need to melt the requirements for VR/AR into its core specifications according to Web architecture. In this workshop, we would like to share our experience from the development of a mobile AR Web Browser [5,6].

Webizing virtual and augmented reality content

The current model for creating VR/AR applications for Web is mostly based on an evolution of traditional computer graphics programming frameworks based on WebGL. In this model, applications are:

Limited functionality – Although 3D contents can be viewed on Web browsers, they are not integrated seamlessly with the existing Web eco-system because of separated handling with other HTML Elements. Users would want to mashup the existing Web API or widgets (e.g., Google Maps, jQuery) which provide versatile functions and live data feeds to the virtual world they create, or vice versa.
No way to refer the real world: There is no official way to handle the recognition data for objects and places in the real world (e.g., feature descriptors for computer vision-based image matching and tracking) for AR Web applications.
Separated rendering context – Like typical HTML content, VR/AR contents also needs to be aware of rendering context (e.g., viewport size, scale) to apply rule-based dynamic layout (e.g., Media Query) or Level of Detail (LOD). However, 3D context based on WebGL is isolated within Canvas Element along with 2D context, and managed separately from the rendering context of other Elements.

We believe that the foundation of the VR/AR for Web needs to evolve in terms of Webizing [7]. – the basic building blocks and operation mechanisms are organized along with the nature of Web architecture. Contrasting with the traditional computer graphics approach, Webized AR/VR applications are:

Mashable – VR/AR contents and existing Web Elements are encompassed seamlessly within a single Web application. As a result, a HTML Element could be embedded to a 3D object and interact each other. In addition, various external frameworks (e.g., Web of Things, Payments, and jQuery) could be added to VR/AR contents.
Referencing the real world – Like images and videos, AR Web applications can simply declare the real-world entities that authors want to recognize by referencing their URIs, and the recognition data (e.g., feature descriptors) are loaded by the Browser and injected into its internal tracking engine.
Responsive to both virtual and real world – The renderer of browser adapts to one of Page, virtual world, and the real world selectively. The Page model is a traditional browsing metaphor based on a 2D plane to render a HTML document in a screen. For VR, the viewport of browser is regarded as the same of a camera in 3D scene (addressed by WebVR API). For AR, the tracking data of real objects and places are applied to the rendering context of browser.

Requirements and opportunities for standardization

To realize the Webized VR/AR content vision outlined above, there are several requirements and opportunities for standardization as follows:

Evolution of HTML for VR/AR – To mingle VR/AR contents and existing Web contents together, it is required for VR/AR contents to become a first-class citizen in HTML. To achieve this, a declarative 3D specification which is fully-compliant with HTML is required. VR/AR contents should be treated in the same manner with typical HTML Elements in a single DOM tree by Browsers.
“Physical Things” as Resources – To enable AR recognition on the Web, all target real-world entities including physical objects and places are regarded as Resource and should be accessible via standard Universal Resource Identifiers (URIs) to provide feature descriptors for identifying and tracking. In our implementation, they are declared as value for the specified CSS property in AR Web application.
New Media Type for VR/AR – The separation of rendering contexts among VR/AR scene and Page could be addressed in terms of the Responsive Web. The different renderers could be dynamically allocated by adding New CSS Media Type for VR/AR or even unified renderer for Mixed Reality (MR). CSS Media Query contains Media Type specification to apply different style sheets to various rendering methods (e.g., speech, braille, embossed, etc.) In our implementation, we have defined the “place” media and AR rendering mode and vision tracking engine are triggered when it is declared in a HTML document.

Conclusion

The VR/AR for Web promises to be one of the most fascinating technology challenges. The scale and overall implications of this nascent field are staggering. We believe that the standardization effort will be crucial to its success and the ability to scale it to its full potential. There are many areas for standardization in the VR/AR for Web, and we are very eager to participate and contribute to this workshop. We believe that one of the critical domains to cover is the definition of a standard development, deployment and management model for VR/AR Web applications, including the formal definition of physical things, media types and the associated data types.

Webizing Virtual and Augmented Reality Content

Introduction

Webizing virtual and augmented reality content

Requirements and opportunities for standardization

Conclusion

References