WebXR Hand Input Module - Level 1

1. Introduction

2. Initialization

If an application wants to view articulated hand pose information during a session, the session MUST be requested with an appropriate feature descriptor. The string "hand-tracking" is introduced by this module as a new valid feature descriptor for articulated hand tracking.

The "hand-tracking" feature descriptor should only be granted for an XRSession its XR device has physical hand input sources that supports hand tracking.

The user agent MAY gate support for hand based XRInputSources based upon this feature descriptor.

NOTE: This means that if an XRSession does not request the "hand-tracking" feature descriptor, the user agent may choose to not support input controllers that are hand based.

3. Physical Hand Input Sources

An XRInputSource is a physical hand input source if it tracks a physical hand. A physical hand input source supports hand tracking if it supports reporting the poses of one or more skeleton joints defined in this specification.

Physical hand input sources MUST include the input profile name of "generic-hand-select" in their profiles.

3.1. XRInputSource

partial interface XRInputSource {
   readonly attribute XRHand? hand;
};

The hand attribute on a physical hand input source that supports hand tracking will be an XRHand object giving access to the underlying hand-tracking capabilities. hand will have its input source set to this.

If the XRInputSource belongs to an XRSession that has not been requested with the "hand-tracking" feature descriptor, hand MUST be null.

3.2. Skeleton Joints

A physical hand input source is made up of many skeleton joints.

A skeleton joint for a given hand can be uniquely identified by a skeleton joint index, which is a nonnegative integer.

A skeleton joint may have an associated bone that it is named after and used to orient its -Z axis. The associated bone of a skeleton joint is the bone that comes after the joint when moving towards the fingertips. The tip and wrist joints have no associated bones.

A skeleton joint has a radius which is the radius of a sphere placed at its center so that it roughly touches the skin on both sides of the hand.

This specification defines the following skeleton joints:

3.3. XRHand

[Exposed=Window]
interface XRHand {
    iterable<XRJointSpace>;
    readonly attribute unsigned long length;
    getter XRJointSpace joint(unsigned long jointIndex);

    const unsigned long WRIST = 0;

    const unsigned long THUMB_METACARPAL = 1;
    const unsigned long THUMB_PHALANX_PROXIMAL = 2;
    const unsigned long THUMB_PHALANX_DISTAL = 3;
    const unsigned long THUMB_PHALANX_TIP = 4;

    const unsigned long INDEX_METACARPAL = 5;
    const unsigned long INDEX_PHALANX_PROXIMAL = 6;
    const unsigned long INDEX_PHALANX_INTERMEDIATE = 7;
    const unsigned long INDEX_PHALANX_DISTAL = 8;
    const unsigned long INDEX_PHALANX_TIP = 9;

    const unsigned long MIDDLE_METACARPAL = 10;
    const unsigned long MIDDLE_PHALANX_PROXIMAL = 11;
    const unsigned long MIDDLE_PHALANX_INTERMEDIATE = 12;
    const unsigned long MIDDLE_PHALANX_DISTAL = 13;
    const unsigned long MIDDLE_PHALANX_TIP = 14;

    const unsigned long RING_METACARPAL = 15;
    const unsigned long RING_PHALANX_PROXIMAL = 16;
    const unsigned long RING_PHALANX_INTERMEDIATE = 17;
    const unsigned long RING_PHALANX_DISTAL = 18;
    const unsigned long RING_PHALANX_TIP = 19;

    const unsigned long LITTLE_METACARPAL = 20;
    const unsigned long LITTLE_PHALANX_PROXIMAL = 21;
    const unsigned long LITTLE_PHALANX_INTERMEDIATE = 22;
    const unsigned long LITTLE_PHALANX_DISTAL = 23;
    const unsigned long LITTLE_PHALANX_TIP = 24;
};

Every XRHand has an associated input source, which is the physical hand input source that it tracks.

Each XRHand has a list of joint spaces which is a list of XRJointSpaces corresponding to each skeleton joint defined in this specification. These all will have their hand set to this.

If an individual device does not support a joint defined in this specification, it MUST emulate it instead.

The list of joint spaces MUST NOT change over the course of a session.

The length attribute MUST return the number 25

3.4. XRJointSpace

[Exposed=Window]
interface XRJointSpace: XRSpace {};

The native origin of an XRJointSpace is the position and orientation of the underlying joint.

The native origin of the XRJointSpace may only be reported when native origins of all other XRJointSpaces on the same hand are being reported. When a hand is partially obscured the user agent MAY emulate the obscured joints, or it MAY report null poses for all of the joints.

Note: This means that when fetching poses you will either get an entire hand or none of it.

This by default precludes faithfully exposing polydactyl/oligodactyl hands, however for fingerprinting concerns it will likely need to be a separate opt-in, anyway. See Issue 11 for more details.

The native origin has its -Y direction pointing perpendicular to the skin, outwards from the palm, and -Z direction pointing along their associated bone, away from the wrist.

For tip skeleton joints where there is no associated bone, the -Z direction is the same as that for the associated distal joint, i.e. the direction is along that of the previous bone. For wrist skeleton joints the -Z direction SHOULD point roughly towards the center of the palm.

Every XRJointSpace has an associated hand, which is the XRHand that created it.

Every XRJointSpace has an associated joint index, which is the joint index corresponding to the joint it tracks.

Every XRJointSpace has an associated joint, which is skeleton joint corresponding to its joint index.

4. Frame Loop

4.1. XRFrame

partial interface XRFrame {
    XRJointPose? getJointPose(XRJointSpace joint, XRSpace baseSpace);
    boolean fillJointRadii(sequence<XRJointSpace> jointSpaces, Float32Array radii);

    boolean fillPoses(sequence<XRSpace> spaces, XRSpace baseSpace, Float32Array transforms);
};

4.2. XRJointPose

An XRJointPose is an XRPose with additional information about the size of the skeleton joint it represents.

[Exposed=Window]
interface XRJointPose: XRPose {
    readonly attribute float radius;
};

The radius attribute returns the radius of the skeleton joint in meters.

The user-agent MUST set radius to an emulated value if the XR device does not have the capability of determining this value, either in general or in the current animation frame (e.g. when the skeleton joint is partially obscured).

5. Privacy & Security Considerations

Since this feature returns new sensor data, the User Agent MUST ask for explicit consent from the user at session creation time.

Data returned from this API, MUST NOT be so specific that one can detect individual users. If the underlying hardware returns data that is too precise, the User Agent MUST anonymize this data (ie by adding noise or rounding) before revealing it through the WebXR Hand Input API.

This API is only supported in XRSessions created with XRSessionMode of "immersive-vr" or "immersive-ar". "inline" sessions MUST not support this API.