Base Types
One of the first dependencies for a Declarative3D approach is the existence of complex data types. Here the vector, matrix and quaternion math are essential while other types like a type for colors would gain the general productivity. Complex base types are essential for 3D operations in computer graphics. The fact that geometric operations like an affine transformation in 3D space depends on a matrix of 4x4 = 16 elements leads to the fact, that it is a logical need to package the data and also the operations into a complex data type.
Other groups (SVG, CSS 3D Transform) have dealt with the absence of linear algebra in HTML development and have defined their own subset of such mathematics. There are also some JavaScript libraries (glMatrix, webgl-mjs, Sylvester) which handle the mathematics in runtime code. The fact that different areas besides 3D computer graphics have already shown the need of such mathematic constructs, leads us to claim the idea, that linear algebra mathematics and the dependent data types could also be declared outside the Declarative3D group for a W3C wide standard.
Use cases
It' a very valid question to ask why we need the access to internal scene graph information at all. Since we target is a declarative system, the answer is: It shouldn't be necessary for the majority of use cases. On the other hand, following the principle of: "Make simple things simple, but complex things possible", we can achieve more flexibility and a leaner standard by providing basic scene graph information.
Here are some use cases, where access to information such as transformation & view/camera matrices would be interesting:
- Projection of (relative) 3D coordinates into screen space (transformation, camera)
- Keep orientation of an object fixed to a reference point, which could be the current camera (aka Billboarding) but also to arbitrary other objects in 3D space
This section should link to our use case section as soon as we have real use cases for functionality above
Base data types
There should be complex base data types, defined for geometric 2D and 3D and also for the color space.
These are:
- Vector2D, Vector3D, Vector4D
- Matrix2D, Matrix3D, Matrix4D
- Quaternion
- ColorRGB, ColorRGBA
Vector, Quaternion & Matrix mathematics should handle operations that are needed for 2D or 3D geometric operations. The available functions could be related to these defined in GLSL for geometric functions. More complex functions defined by interfaces like BLAS (Basic Linear Algebra Subprograms) would be overkill.
For performance reason the definition and implementation should take care of performance issues:
- Copy instructions should be minimized.
- Interfaces should only work on references.
- The base types should internally build upon TypedArrays and reflect these raw containers to the outside.
Another open point for discussion is the availability of array, build upon the addressed data types and the operations on such arrays of complex data types. While with TypedArrays it is possible to define a view into a TypedArray and operate on this subarray. SVG also defines such a type of array, called SVGPointLis & SVGTransformList. SIMD instructions on such an array are important for vertex based operations on the CPU like animation, morphing or blending.
Related technologies
WebGL
Graphic API’s like OpenGL and DirectX have been using vector and matrix math for years. By the fact that WebGL is based on the mobile branch of OpenGL (OpenGL ES) it slightly introduces vectors and matrices into the HTML world. While WebGL natively does not support any library for mathematic operations itself, it uses typed arrays to transfer data to the GPU by Uniforms and attributes. On the other hand, the shading language itself, in this case GLSL, supports vectors and matrices as native basic types a set of geometric functions for arithmetic mathematics on this types.
CSS
CSS 3D Transform brings the ability for 3D transformation of 2D elements from within the webpage. The specification includes a single matrix type: CSSMatrix but not a full set of basic complex types. The system is not designed for the use outside the context of transforming DOM elements.
SVG
Another system recently entered the HTML world, which already uses complex data types is SVG. While their concept is based on a planar system and vector graphics, they have primary defined basis types for work in a 2D system. Beside the definition of a simple 2D point there is also a matrix for affine transformation operations. Another interesting data type they introduced is a type for colors with operations to convert between RGB and ICC. This system is also not designed for general linear algebra mathematics, because the functions are trimmed to a 2D space. Another drawback is that this system is not designed to handle multiple arithmetic operations:
- For a single operation like ‘’the translation of a point’’ the system is sufficient.
- For the reuse of the output data from this operation, the system is insufficient: The output can’t be used directly for a second operation. As an example, the input signature of a translation operation is SVGMatrix.translate(in float x, in float y) and therefore cannot handle complex data types like a SVGPoint as input.
X3D
X3D defines a full set of base types including JavaScript/ECMAScript interfaces.
The X3D DTDs and XML Schemas also provide a means to perform validation of typed data. Further capabilities or definitions in XML Schema would be helpful to allow direct validation of numeric arrays. Current practices rely on use of regular expressions (regexes) to ensure data validity, which tends to be a computationally expensive approach.
In general, the use of X3D base types allows authors to avoid the need for complex matrix mathematics. Instead scenes are typically defined using position and rotation information, which is simpler to compute and compose. This is a declarative approach. The use of matrix mathematics is also available as an option for authors who prefer to embed imperative computational scripts in their scenes.
W3C CSS-SVG Effects Task Force
Matrix4x4
Task force is working on a Matrix4x4 that is usable in CSS, SVG and WebGL.
Current proposal:
// CONSTRUCTORS
// -----------------------------------------------------------------------
// -----------------------------------------------------------------------
[
// Creates an identity matrix
Constructor(),
// Creates a new matrix from the value passed in (which is CSS transform string with
// matrix() or matrix3d() - not other functions).
Constructor(in DOMString string),
// Creates a new matrix from the Float32Array value passed in (length of 6 for a 2d matrix, or 16 for a 4x4 matrix).
Constructor(in Float32array array),
// Creates a new matrix identical to the passed matrix.
Constructor(in Matrix4x4 matrix),
// Creates a new matrix from the array of 6 or 16 values passed in
Constructor(in sequence<float> array)
]
interface Matrix4x4 {
// These attributes are simple aliases for certain elements of the 4x4 matrix
attribute double a; // alias for m11
attribute double b; // alias for m12
attribute double c; // alias for m21
attribute double d; // alias for m22
attribute double e; // alias for m41
attribute double f; // alias for m42
attribute double m11;
attribute double m12;
attribute double m13;
attribute double m14;
attribute double m21;
attribute double m22;
attribute double m23;
attribute double m24;
attribute double m31;
attribute double m32;
attribute double m33;
attribute double m34;
attribute double m41;
attribute double m42;
attribute double m43;
attribute double m44;
// -----------------------------------------------------------------------
// TRANSFORMATION FUNCTIONS
// -----------------------------------------------------------------------
// Multiply itself by a otherMatrix, on the right (this = this * otherMatrix)
void multiply(in Matrix4x4 otherMatrix);
// Multiply this matrix by otherMatrix, on the right (result = this * otherMatrix)
[Immutable] Matrix4x4 multipliedBy(in Matrix4x4 otherMatrix);
// Multiply itself by a otherMatrix, on the left (this = otherMatrix * this)
void leftMultiply(in Matrix4x4 otherMatrix);
// Invert this matrix. Throw an exception if the matrix is not invertible
void invert() raises (DOMException)
// Return the inverse of this matrix. Throw an exception if the matrix is not invertible
[Immutable] Matrix4x4 inverseMatrix() raises (DOMException);
// Translate this matrix in place using the passed values.
// Undefined or NaN parameters will use a value of 0. This allows the 3D form to used for 2D operations.
void translate(in double x,
in double y,
in double z);
// Return a new matrix that is this matrix translated by the passed values.
// Undefined or NaN parameters will use a value of 0. This allows the 3D form to used for 2D operations
[Immutable] Matrix4x4 translatedBy(in double x,
in double y,
in double z);
// Scale this matrix in place using the passed values.
// Passing undefined or NaN for scaleY uses the value of scaleX.
// Passing undefined or NaN for scaleZ uses the value 1.0.
void scale(in double scaleX,
in double scaleY,
in double scaleZ);
// Return a new matrix that is this matrix scaled by the passed values.
// Passing undefined or NaN for scaleY uses the value of scaleX.
// Passing undefined or NaN for scaleZ uses the value 1.0.
[Immutable] Matrix4x4 scaledBy(in double scaleX,
in double scaleY,
in double scaleZ);
// Rotate this matrix in place using the passed values. Each parameter defines
// an angle of rotation around the (corresponding) X, Y or Z axes.
// Passing undefined or NaN for rotY and rotZ instead rotates around the Z axis
// (i.e. rotX = 0, rotY = 0, rotZ = <input rotX>)
// Input values are in radians.
void rotate(in double rotX,
in double rotY,
in double rotZ);
// Return a new matrix that is this matrix rotated by the passed values. Each parameter
// defines an angle of rotation around the (corresponding) X, Y or Z axes.
// Passing undefined or NaN for rotY and rotZ instead rotates around the Z axis
// (i.e. rotX = 0, rotY = 0, rotZ = <input rotX>)
// Input values are in radians.
[Immutable] Matrix4x4 rotatedBy(in double rotX,
in double rotY,
in double rotZ);
// Rotate this matrix in place using the passed values. The x, y and z values
// define a vector about which the matrix is rotated.
// The value of angle is in radians.
void rotateAxisAngle(in double x,
in double y,
in double z,
in double angle);
// Return a new matrix that is this matrix rotated using the passed values. The x, y and z values
// define a vector about which the matrix is rotated.
// The value of angle is in radians.
[Immutable] Matrix4x4 rotatedByAxisAngle(in double x,
in double y,
in double z,
in double angle);
// -----------------------------------------------------------------------
// CONVERSION FUNCTIONS
// -----------------------------------------------------------------------
// Copy the matrix elements into a Float32Array
// Throws an exception if the input is invalid (e.g. wrong length, etc).
void copyIntoFloat32Array(inout Float32Array array) raises (DOMException);
// return a Float32Array (length 16) with the values [m11, m12, ...]
Float32Array toFloat32Array();
// Returns a string that is in the format of a CSS transform.
// (e.g. "matrix3d(m11, m12, ...)")
DOMString toString();
};
XML Schema XSD 1.1
W3C XML Schema Definition Language (XSD) 1.1 Part 2: Datatypes is under review.
We should comment. Upon first inspection, there does not seem to be validation support for defining arrays of floating-point primitive arrays (e.g. MFRotation array of rotation values).
X3D schema definitions accomplish this task by using regular expressions (regexes) to define precise type checking of such arrays. This works but can be memory intensive for some applications when validating large arrays of values.
This section might have what we are looking for:
2.4.2.2 Construction by List A ·list· datatype can be ·constructed· from another datatype (its ·item type·) by creating a ·value space· that consists of finite-length sequences of zero or more values of its ·item type·. Datatypes so ·constructed· have anySimpleType as their ·base type·. Note that since the ·value space· and ·lexical space· of any ·list· datatype are necessarily subsets of the ·value space· and ·lexical space· of anySimpleType, any datatype ·constructed· as a ·list· is a ·restriction· of its base type.