3D-specific DOM Events

From Declarative 3D for the Web Architecture

Mouse3DEvent

Overview

In HTML, interaction is mainly performed with mouse input devices. The DOM Level 2 Events Specification defines a Mouse Event module for this purpose. All events defined in the Mouse Event module generate an event with the MouseEvent interface. This interface provides specific contextual information associated with Mouse events.

The behaviour for 3D objects (with a spatial extend) described within the HTML extension is the same as for 2D HTML events. For 3D interaction, additional 3D specific contextual information is necessary. The Mouse3DEvent extends the DOM MouseEvent to provide this additional information. Beside retrieval of the target object and 2D context information, the position in 3D, the related primitive, the position on the related primitive and its associated vertex attributes are of interest.

We propose a three-step approach with the design principle: Simple things simple, complex things possible.

  1. Provide information that is always available and useful for the most applications directly in the interface for convienence.
  2. In most use cases the vertex attributes for the picked position interpolated in object space is needed, e.g. the interpolated normal. Provide a way to query and access this data.
  3. Some use cases might require other interpolation methods, neighborhood information etc. Provide a way to access this specific data.

For access to attribute data, the input data is the relevant source. This is important, because in CG with its general APIs and programmable shaders it's not possible to rely on what data the shader actually uses for the shading process. Nevertheless, for optimizations purposes and known fixed-function shaders, the agent might compute some of the information, e.g. the interpolated vertex attributes on the GPU.

Proposal

interface Mouse3DEvent : MouseEvent {
 // position in world space
 readonly Vec3 worldPosition;

 // position in target’s local space
 readonly Vec3 localPosition; 
 
 // linear interpolated in object space
 readonly NamedVertexAttributeMap attributes;
 
 // barycentric coordinates of the intersection point
 readonly Vec3 barycentricCoords;
 
 // Identifier to the picked primitive. The picked geomerty provides
 // the API to resolve more detailed information on the primitive
 readonly int primitiveId;
 
}
interface NamedVertexAttributeMap {
 getter TypedArray (in DOMString name);
}

Explanation

The interface fits to the proposed three-step approach:

  1. The only data that is alway available and of special interest is worldPosition and localPosition, available as 3-component vector. This provides convient access to the most important data.
  2. All other attributes can be queried using the NamedVertexAttributeMap. The query will return null for attributes that are not available and a TypedArray for available vertex attributes. The returned value is linear interpolated in object space.
  3. If the access through 2 is not sufficient e.g. because an other interpolation method is needed, the user has the possibility to access the vertex attribute through the picked object. For this purpose, the interface provides an index and barycentric coordinates of the picking. The picked object provides an interface to retrieve the picked primitive from the index (yet to be defined).

Usage example

Usage example for the NamedVertexAttributeMap:

function mouseClicked(evt) {
 var color = evt.attributes['color'];
 // or equivalent
 color = evt.attributes.color;
 assert(color.length == 3); // user has to know the size of the attribute
 alert("Color is: " + color[0] + " " + color[1] + " " + color[2])
}