1. Introduction
Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The WebXR Augmented Reality module expands the functionality available to developers when their code is running on AR hardware.
1.1. Terminology
Augmented Reality describes a class of XR experiences in which virtual content is aligned and composed with the real-world environment before being displayed to users.XR hardware can be divided into categories based on display technology.
-
Devices described as additive light, also known as see-through, use transparent optical displays to present virtual content. On these devices, the user may always be able to see through to the real-world environment regardless of developer requests during session creation.
-
Devices described as pass-through use an opaque display to combine virtual content with a camera stream of the real-world environment. On these devices, the real-world environment will only be visible when the developer has made an explicit request for it during session creation.
-
Devices described as opaque fully obscure the real-world environment and do not provide a way to view the real-world environment.
2. WebXR Device API Integration
2.1. XRSessionMode
As defined in the WebXR Device API categorizes XRSession
s based on their XRSessionMode
. This module expands the allowable values of the XRSessionMode
enum to include "immersive-ar"
.
enum {
XRSessionMode ,
"inline" ,
"immersive-vr" "immersive-ar" };
A session mode of "immersive-ar"
indicates that the session’s output will be given exclusive access to the immersive XR device display and that content is intended to be blended with the real-world environment.
The definition of immersive session is also expanded to include sessions with a mode of "immersive-ar"
. On compatible hardware, user agents MAY support "immersive-vr"
sessions, "immersive-ar"
sessions, or both. Supporting for the additional "immersive-ar"
session mode, does not change the requirement that user agents MUST support "inline"
sessions.
NOTE: This means that "immersive-ar"
sessions support all the features and reference spaces that "immersive-vr"
sessions do, since both are immersive sessions.
"immersive-ar"
sessions are supported.
navigator. xr. supportsSession( 'immersive-ar' ). then(() => { // 'immersive-ar' sessions are supported. // Page should advertise AR support to the user. }
"immersive-ar"
XRSession
.
let xrSession; navigator. xr. requestSession( "immersive-ar" ). then(( session) => { xrSession= session; });
2.2. XREnvironmentBlendMode
When drawing XR content, it is often useful to understand how the rendered pixels will be blended by the XR Compositor with the real-world environment.enum {
XREnvironmentBlendMode "opaque" ,"alpha-blend" ,"additive" };partial interface XRSession { // Attributesreadonly attribute XREnvironmentBlendMode environmentBlendMode ; };
The environmentBlendMode
attribute MUST report the XREnvironmentBlendMode
value that matches blend technique currently being performed by the XR Compositor.
-
A blend mode of
opaque
MUST be reported if the XR Compositor is using opaque environment blending. -
A blend mode of
alpha-blend
MUST be reported if the XR Compositor is using alpha-blend environment blending. -
A blend mode of
additive
MUST be reported if the XR Compositor is using additive environment blending.
2.3. XR Compositor Behaviors
When presenting content to the XR device, the XR Compositor MUST apply the appropriate blend technique to combine virtual pixels with the real-world environment. The appropriate technique is determined based on the XR device's display technology and the mode.
-
When performing opaque environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing exclusively 100% opaque black pixels. The composited output is then presented on the XR device. This technique MUST be applied on opaque and pass-through displays when the mode is set to either
"immersive-vr"
or"inline"
. This technique MUST NOT be applied when the mode is set to"immersive-ar"
, regardless of the XR Device's display technology. -
When performing alpha-blend environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing pixel representations of the real-world environment. These pixel representations must be aligned on each
XRFrame
to thetransform
of each view inviews
. The composited output is then presented on the XR device. This technique MUST be applied on pass-through displays when the mode is set"immersive-ar"
. This technique MUST NOT be applied when the mode is set to"immersive-vr"
or"inline"
regardless of the XR Device's display technology. -
When performing additive environment blending, the rendered buffers obtained by the XR Compositor are composited using lighter blending before being presented on the XR device. This technique MUST be applied on additive light displays, regardless of the mode.
The XR Compositor MAY make additional color or pixel adjustments to optimize the experience. The timing of composition MUST NOT depend on the blend technique or source of the real-world environment. but MUST NOT perform occlusion based on pixel depth relative to real-world geometry; only rendered content MUST be composed on top of the real-world background.
NOTE: Future modules may enable automatic or manual pixel occlusion with the real-world environment.
The XR Compositor MUST NOT automatically grant the page access to any additional information such as camera intrinsics, media streams, real-world geometry, etc.
NOTE: Developers may request access to an XR Device's camera, should one be exposed through the existing [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/) specification. However, doing so does not provide a mechanism to query the XRRigidTransform
between the camera’s location and the native origin of the viewer reference space. It also does not provide a guaranteed way to determine the camera intrinsics necessary to match the view of the real-world environment. As such, performing effective computer vision algorithms wil be significantly hampered. Future modules or specifications may enable such functionality.
3. Security, Privacy, and Comfort Considerations
3.1. Protected functionality
add information about new threats and mitigations introduced by functionality in this module
4. Acknowledgements
The following individuals have contributed to the design of the WebXR Device API specification:
-
Sebastian Sylvan (Formerly Microsoft)
And a special thanks to Vladimir Vukicevic (Unity) for kick-starting this whole adventure!