WebXR Augmented Reality Module

Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

The Immersive Web Working Group maintains a list of all bug reports that the group has not yet addressed. This draft highlights some of the pending issues that are still to be discussed in the working group. No decision has been taken on the outcome of these issues including whether they are valid. Pull requests with proposed specification text for outstanding issues are strongly encouraged.

This document was published by the Immersive Web Working Group as a Working Draft. This document is intended to become a W3C Recommendation.

This document is a First Public Working Draft.

Publication as a First Public Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 March 2019 W3C Process Document.

This WebXR Argumented Reality Module is designed as a module to be implemented in addition to WebXR Device API, and is originally included in WebXR Device API which was divided into core and modules.

1. Introduction

Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The WebXR Augmented Reality module expands the functionality available to developers when their code is running on AR hardware.

1.1. Terminology

Augmented Reality describes a class of XR experiences in which virtual content is aligned and composed with the real-world environment before being displayed to users.

XR hardware can be divided into categories based on display technology.

Devices described as additive light, also known as see-through, use transparent optical displays to present virtual content. On these devices, the user may always be able to see through to the real-world environment regardless of developer requests during session creation.
Devices described as pass-through use an opaque display to combine virtual content with a camera stream of the real-world environment. On these devices, the real-world environment will only be visible when the developer has made an explicit request for it during session creation.
Devices described as opaque fully obscure the real-world environment and do not provide a way to view the real-world environment.

2. WebXR Device API Integration

2.1. XRSessionMode

As defined in the WebXR Device API categorizes XRSessions based on their XRSessionMode. This module expands the allowable values of the XRSessionMode enum to include "immersive-ar".

enum XRSessionMode {
  "inline",
  "immersive-vr",
  "immersive-ar"
};

A session mode of "immersive-ar" indicates that the session’s output will be given exclusive access to the immersive XR device display and that content is intended to be blended with the real-world environment.

The definition of immersive session is also expanded to include sessions with a mode of "immersive-ar". On compatible hardware, user agents MAY support "immersive-vr" sessions, "immersive-ar" sessions, or both. Supporting for the additional "immersive-ar" session mode, does not change the requirement that user agents MUST support "inline" sessions.

NOTE: This means that "immersive-ar" sessions support all the features and reference spaces that "immersive-vr" sessions do, since both are immersive sessions.

The following code checks to see if "immersive-ar" sessions are supported.

navigator.xr.supportsSession('immersive-ar').then(() => {
  // 'immersive-ar' sessions are supported.
  // Page should advertise AR support to the user.
}

The following code attempts to retrieve an "immersive-ar" XRSession.

let xrSession;

navigator.xr.requestSession("immersive-ar").then((session) => {
  xrSession = session;
});

2.2. XREnvironmentBlendMode

When drawing XR content, it is often useful to understand how the rendered pixels will be blended by the XR Compositor with the real-world environment.

enum XREnvironmentBlendMode {
  "opaque",
  "alpha-blend",
  "additive"
};

partial interface XRSession {
  // Attributes
  readonly attribute XREnvironmentBlendMode environmentBlendMode;
};

The environmentBlendMode attribute MUST report the XREnvironmentBlendMode value that matches blend technique currently being performed by the XR Compositor.

A blend mode of opaque MUST be reported if the XR Compositor is using opaque environment blending.
A blend mode of alpha-blend MUST be reported if the XR Compositor is using alpha-blend environment blending.
A blend mode of additive MUST be reported if the XR Compositor is using additive environment blending.

2.3. XR Compositor Behaviors

When presenting content to the XR device, the XR Compositor MUST apply the appropriate blend technique to combine virtual pixels with the real-world environment. The appropriate technique is determined based on the XR device's display technology and the mode.

When performing opaque environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing exclusively 100% opaque black pixels. The composited output is then presented on the XR device. This technique MUST be applied on opaque and pass-through displays when the mode is set to either "immersive-vr" or "inline". This technique MUST NOT be applied when the mode is set to "immersive-ar", regardless of the XR Device's display technology.
When performing alpha-blend environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing pixel representations of the real-world environment. These pixel representations must be aligned on each XRFrame to the transform of each view in views. The composited output is then presented on the XR device. This technique MUST be applied on pass-through displays when the mode is set "immersive-ar". This technique MUST NOT be applied when the mode is set to "immersive-vr" or "inline" regardless of the XR Device's display technology.
When performing additive environment blending, the rendered buffers obtained by the XR Compositor are composited using lighter blending before being presented on the XR device. This technique MUST be applied on additive light displays, regardless of the mode.

The XR Compositor MAY make additional color or pixel adjustments to optimize the experience. The timing of composition MUST NOT depend on the blend technique or source of the real-world environment. but MUST NOT perform occlusion based on pixel depth relative to real-world geometry; only rendered content MUST be composed on top of the real-world background.

NOTE: Future modules may enable automatic or manual pixel occlusion with the real-world environment.

The XR Compositor MUST NOT automatically grant the page access to any additional information such as camera intrinsics, media streams, real-world geometry, etc.

NOTE: Developers may request access to an XR Device's camera, should one be exposed through the existing [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/) specification. However, doing so does not provide a mechanism to query the XRRigidTransform between the camera’s location and the native origin of the viewer reference space. It also does not provide a guaranteed way to determine the camera intrinsics necessary to match the view of the real-world environment. As such, performing effective computer vision algorithms wil be significantly hampered. Future modules or specifications may enable such functionality.