Information

WebSpatial API for Spatialized HTML/CSS and Spatialized PWAs on spatial and multimodal AI devices
  • Upcoming
  • Tentative
  • Breakout Sessions

Meeting

Event details

Date:
– Japan Standard Time
Status:
Tentative
Location:
R01
Participants:
Dexter Yang
Big meeting:
TPAC 2025 (Calendar)

The strong demand for multimodal interaction capabilities in client-side AI is driving the evolution of next-generation general computing platforms from screen-based and handheld devices to hands-free wearable devices, such as augmented reality glasses and mixed reality visors (head-mounted displays).

Apps (include Web) on such next-gen platforms must be able to:

  1. Fully leverage 3D space surrounding the user, moving beyond the limitations of 2D planes, breaking free from window borders.

  2. Work with a new OS architecture based on Unified Rendering and Shared Space, in which apps no longer render themselves independently, they can blend in with other apps in the same shared space, and use OS-managed spatial containers instead of old-style windows or browser tabs.

  3. Integrate seamlessly with the real-world environment, for example, Apple's recently proposed liquid/frosted glass UI. Instead of using a fixed solid background color or just switching between light and dark modes, the UI dynamically renders a blended material background based on what's behind each part of the interface from the user's perspective, making it clear and readable no matter the lighting or colors in the environment.

  4. In addition to conventional 2D panel-based and 3D canvas-based GUIs, apps should be able to include real volumetric 3D content that can be viewed from all angles and measured in real-world units.

  5. Whether 2D or 3D, elements must be capable of positioning and transformation along the spatial Z-axis, supporting spatial interaction events to acquire positional and motion data within 3D space, thereby enabling interactions such as dragging, rotation, and scaling.

  6. Don't break away from mainstream GUI development ecosystems, keep using the established view components, layout systems, and styling systems from 2D GUI development (for the web, that means HTML/CSS). In most application scenarios, limited 3D development requirements should be embedded within the overarching 2D development approach. For example, 3D container elements should be able to join the X/Y layout system just like 2D elements, while supporting full 3D-engine-style development inside. That means in HTML code you can use different Entities (different geometric Entities, Entities capable of loading 3D model assets, or Entities capable of hosting 2D HTML or particle effects), declare, load and configure materials and textures, and work with EntityTransform and physics engine properties.

Within the existing Immersive Web WG, the WebXR standard, analogous to OpenXR, always take over the binocular display and the entire spatial environment, perform self-contained rendering, and build interactions from scratch, thereby failing to satisfy the aforementioned requirements.

There are some scattered Spatial Web features right now, like Model Element, Immersive Media (Fullscreen API), and Interactive Regions (Hover Effects for Eye-Hand Interactions). They can't systematically meet the needs above, and they still lag far behind native spatial apps.

A systematic solution can only be achieved by equipping existing HTML/CSS and PWA technologies with spatial capabilities, extending the current 2D web standards with a minimal set of new APIs (let's call them the WebSpatial API).

Such an approach would enable the open web to gain entry into the paradigm shift of general computing platforms, and potentially emerge as the optimal application solution for these platforms. This is because both AI OS and MR OS inherently demand vast numbers of apps that are installation-free, on-demand, and disposable - characteristics naturally aligned with the web.

<img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/e5b05ecd-cc8e-4211-b0a8-7b74c322c782" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/d6c421b5-6ff1-476f-ac46-ecea9b0f2995" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/561aa5b4-4a14-4fba-8d1a-7a442ee69789" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/cd699912-136c-4bab-9098-c39c2e074b12" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/6af0ec96-470e-41c6-a0a1-15d6235dbe1b" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/7fcadc18-bb47-4d2b-90f9-01b7049df820" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/d3bc04e8-e02d-443c-8468-f6306cf5e89f" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/ffd3c5e5-fbd3-4ef4-92b9-4d8cee7e3bc7" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/5399d562-0a42-47ac-8d9c-8f561de31fa0" />

Agenda

Chairs:
Dexter Yang

Description:
The strong demand for multimodal interaction capabilities in client-side AI is driving the evolution of next-generation general computing platforms from screen-based and handheld devices to hands-free wearable devices, such as augmented reality glasses and mixed reality visors (head-mounted displays).

Apps (include Web) on such next-gen platforms must be able to:

  1. Fully leverage 3D space surrounding the user, moving beyond the limitations of 2D planes, breaking free from window borders.

  2. Work with a new OS architecture based on Unified Rendering and Shared Space, in which apps no longer render themselves independently, they can blend in with other apps in the same shared space, and use OS-managed spatial containers instead of old-style windows or browser tabs.

  3. Integrate seamlessly with the real-world environment, for example, Apple's recently proposed liquid/frosted glass UI. Instead of using a fixed solid background color or just switching between light and dark modes, the UI dynamically renders a blended material background based on what's behind each part of the interface from the user's perspective, making it clear and readable no matter the lighting or colors in the environment.

  4. In addition to conventional 2D panel-based and 3D canvas-based GUIs, apps should be able to include real volumetric 3D content that can be viewed from all angles and measured in real-world units.

  5. Whether 2D or 3D, elements must be capable of positioning and transformation along the spatial Z-axis, supporting spatial interaction events to acquire positional and motion data within 3D space, thereby enabling interactions such as dragging, rotation, and scaling.

  6. Don't break away from mainstream GUI development ecosystems, keep using the established view components, layout systems, and styling systems from 2D GUI development (for the web, that means HTML/CSS). In most application scenarios, limited 3D development requirements should be embedded within the overarching 2D development approach. For example, 3D container elements should be able to join the X/Y layout system just like 2D elements, while supporting full 3D-engine-style development inside. That means in HTML code you can use different Entities (different geometric Entities, Entities capable of loading 3D model assets, or Entities capable of hosting 2D HTML or particle effects), declare, load and configure materials and textures, and work with EntityTransform and physics engine properties.

Within the existing Immersive Web WG, the WebXR standard, analogous to OpenXR, always take over the binocular display and the entire spatial environment, perform self-contained rendering, and build interactions from scratch, thereby failing to satisfy the aforementioned requirements.

There are some scattered Spatial Web features right now, like Model Element, Immersive Media (Fullscreen API), and Interactive Regions (Hover Effects for Eye-Hand Interactions). They can't systematically meet the needs above, and they still lag far behind native spatial apps.

A systematic solution can only be achieved by equipping existing HTML/CSS and PWA technologies with spatial capabilities, extending the current 2D web standards with a minimal set of new APIs (let's call them the WebSpatial API).

Such an approach would enable the open web to gain entry into the paradigm shift of general computing platforms, and potentially emerge as the optimal application solution for these platforms. This is because both AI OS and MR OS inherently demand vast numbers of apps that are installation-free, on-demand, and disposable - characteristics naturally aligned with the web.

<img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/e5b05ecd-cc8e-4211-b0a8-7b74c322c782" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/d6c421b5-6ff1-476f-ac46-ecea9b0f2995" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/561aa5b4-4a14-4fba-8d1a-7a442ee69789" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/cd699912-136c-4bab-9098-c39c2e074b12" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/6af0ec96-470e-41c6-a0a1-15d6235dbe1b" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/7fcadc18-bb47-4d2b-90f9-01b7049df820" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/d3bc04e8-e02d-443c-8468-f6306cf5e89f" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/ffd3c5e5-fbd3-4ef4-92b9-4d8cee7e3bc7" /> <img width="1920" height="1080" alt="Image" src="https://github.com/user-attachments/assets/5399d562-0a42-47ac-8d9c-8f561de31fa0" />

Goal(s):
Initiate the discussion and drafting of the WebSpatial API's requirements and standardization.

Materials:

Joining Instructions

Instructions are restricted to W3C users . You need to log in to see them.

Export options

Personal Links

Please log in to export this event with all the information you have access to.

Public Links

The following links do not contain any sensitive information and can be shared publicly.

Feedback

Report feedback and issues on GitHub.