Meeting minutes
Spatial APIs for the Web
Slideset: https://www.w3.org/2025/11/WebSpatial_TPAC2025_Breakout_Session.pdf
[introductions from speakers]
[presentation is being recorded]
AI devices are changing interfaces from flat screen into the world around us.
AI glasses integrate information into physical space.
Expecting all apps, not just 3D games, need to consider spatial interaction
[Example of floating UI panels in an office space]
3D Web pages can pull out elements into 3D space, have multiple windows or panels in 3D space
AI agents example: order boba tea from store website. Don't want to install the app.
Apple, Google extending 2D GUI frameworks to 3D experiences.
VisionOS apps and Jetpack apps.
Web should not fall behind.
What we currently have: native, react, mini-apps, each has downsides
closed and platform-exclusive walled gardens
Loss of URL access, no-install on-demand use
No integration with Web ecosystem
Paradigm shift in VR OSes.
Traditionally had non-unified rendering, compositor-based. OS doesn't understand the content a given app is rendering.
App can create fully immersive application by taking over rendering.
But trend towards spatial apps, multiple apps sharing space.
Could have volumetric models, light reflecting off of them.
Applications might have frosted background, showing the environment behind.
This requires unified rendering model, where OS does the rnedering.
App focused on rendering within their container
This allows multiple apps to co-exist. Both VisionOS and ?? are moving to this paradigm.
Also allows mixing 2D and 3D content, interacting in unique ways.
Apple established industry pattern with 2D-3D hybrid GUIs.
The OS is responsible for managing these spatial containers.
[review of interaction modes]
WebXR provides a lot of powerful features, allows high-fidelity 3D games
but need mor
WebXR sessions can't co-exist with other apps, use low-level 3D APIs along with WebGL/WebGPU, so hard for web devs to manage
For HTML and CSS, we have a powerul 2D graphics layout framework
Opportunity to expand this to spatial experiences.
Existing z-index is about stacking order, not z-axis placement; 3D transforms are flatted into 2D projections
HTML elements are 2D flat panels with no volume
Lastly for colors, no materials. Only colors.
Wrt JS APIs, don't have ability to open windows around the user.
Specify size or placement of the window, create volumes
For interaction models, need to go past simple pointer events and add spatial gestures and more complex interaction models.
Some features proposed, like <model>, which is a step towards spatial web
Lot of room for expansion.
Safari has also added an immersive mode
Take an existing 2D app and make it more sptialized, but there are limitations
<model> can render 3D models, but also has limitations. E.g. can't program the model.
Spatial browsing in Safari, doesn't necessarily rely on developer building a 3D website. It spatializes an existing 2D application.
Maybe some websites can be spatialized using AI or heuristics, but wouldn't create a truly spatialized web experience
So while we can do intuitive things like 3D-ifying photos, there are limtations.
We need dedicated APIs and tools for developer to design a truly 3D experience
PWAs, once installed, can run instandalone windows.
This would be benefition to spatial applications, but uninstalled would be constrained by browser tabs.
Third problem, Web 3D is still hard, and doesn't have as many developers as traditional Web developers
Looking at games, you have a game loop
Developer controls all aspects of the game engine.
They can specify all the components, the physics, the animation, etc.
Performance is also an issue to control
2D web development is event-based, developer responds to events that are triggered.
with 3D engines, you have powerful tools, e.g. Unity and Unreal help simplify complexity
But you can't just plug in 2D UIs
Most regular web developers find it hard to enter this 3D wrold.
So using 2D framework with 3D parts.
We need developers to create powerful applications, as they hvae been for 2D Web
Instead of 2D containing 3D elements
Imagine having a shared space managed by OS, and all spatial apps live within their own 2D containers
Then 2D spatial app can create 3D elements to it
Some 3D containers can [missed]
We believe that having 2D development paradigm, that can import or use 3D models would be very powerful.
We've seen some examples, especially with SwiftUI
Basically static 3D models, Model3D View
Also realityView using RealityKit API to build dynamic 3D content
This slide shows the high-level view of this sytem.
2D elements can contain 3D assets, which can contain 2D views.
Another element we have is <canvas> element.
Mainly used for 2D drawing
3D engines for <canvas>
Challenges are that it's mainly focused on dynamic content development
Application can draw to it, but browser/OS doesn't know what that content is.
So it doesn't fit into the unified rendering model.
Developing for <canvas> also isn't very intuitive for web developers, it's a different paradigm.
There are some 3D engines, e.g. Unity and Aframe libraries can help, but it's quite different from traditional web dev
WebXR is bulit on top of <canvas> and uses low-level 3D APIs, not web dev paradigm
We think there needs to be a new paradigm, we call it Web Spatial.
Extend the full power of web platform in 3D environment.
Existing standard Web should be extended spatial development so that we can have both traditional 2D stuff and 3D content co-exist
Expand the existing ecosystem to spatial applications.
Can see something similar in SwiftUI, existing APis are back-compatible, but extedn to 3D
Could be new APIs in HTML, CSS, JS
preserving existing content while extending to 3D
Proposal 1: Depth control in CSS.
Also extend transforms to actually work in 3D
So that rotation is actually in 3D space not just a projection.
Proposal 2: Materials
We need new propertys to define translucency, transpraency, crate floating materials, or other interactions withspatial environment.
Proposal 3: Interaction models
We need spatial events.
We thought of some that would be needed, e.g. spatialclick, spatiadrag, etc.
hgih-level APis as well as low-level
Propsal 4: On PWA front, we can expand existing deployment to allow additional configurations in windows and positioning without sacrificing existing model of install-free apps.
Proposal 5: Spatial Scenes
Initializng scenes in web mainfest, postioning windows with JS
Proposal 6: For containing 3D elements, the <model> can be extended to support multiple assets
and also be able to control it dynamically using JS
Proposal 7: Extend canvas model by creating a <reality> element which has primitives that allow the developer to dynamically build a 3D asset
You can declare boxes, spheres, basica elements, materials,transforms, etc.
This is just the beginning, it's a long road.
Open for discussion now!
Discussion
<ruoya> https://
cabanier: What about the HTML in canvas proposal? Wouldn't that solve some of the problems?
cabanier: it allows you to render HTMl inside of canvas, so can do it-testing, etc.
cabanier: Previously you had to draw rectangles, but now could feed it HTMl and tell it to draw that content.
Siyaman: [missed response]
ada: We've been thinking about this for awhile now, pretty much immediately after <model>
… Don't want to go into too much detail, but we have a pretty well thought-through plan for directing where things should go [?]
… Discussing new capability we've discussed is having multiple <model> s inside an element and composing them otgether.
ada: There are some challenges in bringing spatial to Web
… don't want to break content
… but wdon't want to break web paradigms.
… For example, by allowing ocntent outside the portal, allows content to be rendered outside the window, potentially over URL bar, etc.
… it's a feature that's wanted for spatial web, but I don't think we could just do it naively with transforms
… but htere's definitly room to do it
/shtere/there/
… s/wdon/we dont/
ada: Agree webXR not the model for sptial Web going forward. It has specific use cases.
… HTML in canvas will be useful for producing interfaces for WebXR experiences
… those should be an exclusive one
… But I think there's definitely room for a more ??? solution to exist alongside WebXR
ada: Would like to go into more detail, but no time to do a whole slide deck right now, but hope to discuss more stuff soon within the ?? group!
ruoya: In November 20 we'all have another immersive web meeting, we can talk more then
ada: yes, I should have slides for that!
limin: I'm on MSFT Edge. You mentioned declarative??
… Not famliar with app system on VisionOS, what is that experience like? Can you install 3D apps? How does user access it?
… Are these widgets? How does uer get to it?
Siyaman: Traditional web applicaitons, you go to a URL and you can just use it
… superapps in China like AliPay or WeChat, you have a super app, and you just go to a miniapp and just use it
ruoya: AI agent is also like a superapp. Information comes from the Web
… Web is a very good technology for install-free use cases
Siyaman: One conern we have with wearable AI devices. Ppl might have question, want to order from local restaurant, use the app and make an order and then close it, goes away.
Limin: Wrt miniapps, I believe there's a meeting later this week.
<Zakim> ada, you wanted to talk about the standardisation process
ada: Regarding points of what to do, my plan for standardization for this was to work on it withi nthe CSS WG
… some will also need to go to WHATWG
… for proividing a place to work on it, I'm happy to make a repo in Immersive Web CG
… if CG agrees
… I think it's a good venue for discussion, although it might be standardized elsewhere
… email me or other chairs to help set up discussions
ruoya: Sounds good
adekker: Very excited to see this at a broad level
… I come from a web dev background
… went from traditional web to mobile web, that transition was a slow tarnsition
… not seeing what happens with spatial
… running a session tomorrow wrt transparency
… Was going toask, all of the thing syou talk about are very interesting and useful for web devs
… they also come with their own security risks etc.
… how do you see htis adoption happening over time?
… web spatial sits as its own runtime, as a sandbox
ruoya: [can't hear]
Siyaman: We have one implementation to showcase some of these things. Not necessarily endgame.
… so don't get to bogged down in details
… but we do want it to be part of web community
… and back-compat with existing web applications is a huge benefit
… We want to bring that forard to Web spatial
… Maybe existing browsers can support these applications
… maybe there's a mode
… but I think there's a lot that has to be figured outhere
… Security and privacy aspects to figure out
adekker: Do you have a priority list? Do you see one of these being more important than others?
ruoya: We just launched our ?? first version this year in May, so we are super early, very small community of developers adopting our SDK
… Our current focus is to finish the main APIs within our SDK
… andmake it cross-platform
… For sure we need to consider privacy etc.
… It's all on our roadmap, but we have a long road and need support of the community here.
ruoya: https://
… Here is what we support right now
… It's very easy to start
… and you can create a demo
… Here's a demo, that shows comparison between 2D web and translation into a spatialized app
… we still maintain 2D website, but it has a spatialized features if you load it into an XR device
… Here's another demo, using our SDK you can build demos like this [multiple panels moving around]
… You can use HTML and CSS 2D web tech to make it happen
… Right now we have 7 apps in Apple VisionPro store built with our SDK
[shows video of an app]
… This is a music player app
Siyaman: Using existing web tech, so don't have to learn a whole new API to develop these applications
… can bulid a new app in a few weeks
[video of a shoe store]
ruoya: Early feedback indicates it's easy to use this SDK
ruoya: Will continue taking questions!
[video of minesweeper in 3D]
adekker: Thinking about scope, with the examples you provide, these are scoped to adomain like PWA?
… what's the navigation look like?
… for example, is this a single-page application?
ruoya: Yes
adekker: So can you link between spatial apps?
[explanation of opening new windows and such, didn't catch it]
Siyaman: It's like a PWA app on your phone. Within the app you can navigate back and forth
… normal browser link will open up a new browser window
… Have to check if you can link from one spatial site to another
adekker: That's part of the beauty of the web
ruoya: In our roadmap, we're planning to have web spatial browser
… can open a link as a native app, browser will have sdk already
… will run like the Web
At time.