Meeting minutes
session proposal: w3c/
andrew: welcome, Andrew from Meta Quest Browser, interested in transparency for spatial devices but also good time to think about it for other platforms
adekker: I used to be a web dev, was there pre-iphone and WWDC transformation. From web dev perspective you could start to take advantage of mobile platform
adekker: in the future to take advantage of VR/AR glasses with web, transparency is a big part of it
adekker: in these future devices you don't want giant rectangles, 50 chrome windows getting in the way
adekker: being able to match the native experiences like diekus mentioned. From spatial perspective not distracting users, letting web experiences blend in with the environment. Or in limited FOV, small form factors, not desktop/mobile form factor
adekker: apps can build for spatial devices at the moment but websites can't
adekker: accessibility, secuirty, privacy, usability, interaction, browsability, performance, .... challenges to think about
adekker: current proposals: backdrop-filter (more historical?); environment-blending; window transparency proposal from 3 years ago that was controversial; more recently diekus's explainer around materials in installed web applications
diekus: windows introduced materials like acrylic. Apple introduced materials like ultra thin/thick. Applications started gaining capaiblities to seem more modern, title bar is gone, controls in title bar
diekus: PWAs were lagging behind native apps due to cool effects like transparency. These might be taxing (performance)
diekus: If I wanted to replace the calculator app with ?? effect, I can't do that today
diekus: Can't support every single material from every single platform. Start with few, some that are more translucent
diekus: Thinking about questions like should this be a manifest thing for PWAs? Or more broadly should it be a CSS brush color
adekker: From my POV PWAs solve a lot of issues like scoped URLs, don't need address bar. But also in augmented reality is the whole web going to be 100s of PWAs? Reduces power of the web
adekker: Open question 1: is this a capability that should be unique to PWAs? Or individual webpages in the Browser, which opens up to a lot of security / interaction issues
cabanier: Would be a shame if we limited to only PWAs
diekus: What's the exact use case? The explainer was focused on PWAs because it's about matching native app experience. What is your use case?
adekker: In AR/VR we have Browser windows with a ton of chrome. Constraining how web content can be multitasked with because of the size and chrome of these windows. From developer perspective, web is a beautiful way to create web interfaces without creating an app, uploading to an app store.
adekker: Anyone can create quick dashboards and UIs with web technologies
diekus: Is your end goal to have window frames that are not rectangular because of transparency. Or is it to have design language using translucency. Or for developer to be able to set a translucent color
adekker: It's about letting developers break out of the rectangle in spatial. Like 3D CSS, introduces a ton of security, performance, privacy issues. How do we work towards a spatial web
Brandel: You're talking about transparent windows and spatial CSS, separate but closely related. Practical for many of us with spatial browsers to allow a transparent texture. But also composited ???. In AVP safari we have inline model element which is rendered ??? to compositor. Being able to use stereoscopic buffers will be important
adekker: We can still constrain web content inside the rectangle (bounding box) even if it's not visible
adekker: e.g. if I have a timer website, then multitask with some other stuff. I want that website to minimize itself in a way that's still visible
Brandel: Do you need transparency or just more powerful window management e.g. popups
?: or there's the picture in picture API
but doesn't support transparency today
diekus: The idea of moving the timer window and making it small, is that still a frame? Or you make it transparent and becomes an SVG drawn by itself
diekus: In the ads for AVP you can see some sort of transparency. If you want to build web applications, the frame could be sort of translucent
adekker: PWAs solve a lot of the issues
Mek: How difficult would it be? The site could provide a manifest. What wouludn't be solved by doing this for PWAs
adekker: If you think about what the future of spatial browsing looks like -- is everything going to be a PWA
Mek: The sites that want to use transparency can be a PWA
Mek: If you want to be a standalone thing without Browser chrome, that is a PWA
cabanier: It might be ok to have Browser chrome for just webpages, just reduced chrome
adekker: If you have a tiny widget inside a massive window how do you deal with click-jacking? Per pixel hit testing, etc
adekker: Are there security considerations here for things like iframes?
cabanier: iframes are already transparent
Brandel: window.open has a pretty unstructed set of parameters. In the 90s people could specify lots of things. You can specify the size of the opened window. VisionOS inherits from ipadOs, we are handling newwindow requests as new tabs. If you wanted to write a UA that honored new parameters that shouldn't be a problem
Mek: We added custom parameters to it before, you could do that too
diekus: If I were to build an application for Quest / VisionPro, if I create an empty application from template, would it have transparency
adekker: No on Quest
diekus: If there was a webview you could add to native frame, and you set the background of the web content to transparent, could that work?
diekus: You would need to enable a hint, if the platform supports it then great, otherwise just the normal color
Brandel: Out of the box Webkit/WebView is going to draw onto an opaque buffer. Has the ability to manage transparent buffers. One of the things we hit when doing model element is things are P3 linear colorspace, but things need to in sRGB space, so there were coloring disparities. Next week we'll talk more about compositing color modes
Brandel: Might not be possible without modifying webkit to do it
<diekus> Link to the explainer: https://
Brandel: Currently don't have PWAs but they would have different set of priviliges. Translucent materials isn't webkit that's RealityKit, different renderer
diekus: Question for Steve: if you have a micro frame on Windows and put a webview in it, make the background transparent, is that possible? Bad idea?
Steve: sure it's possible. We did a lot of work to enable micah on Windows. To integrate it with chromium compositor
Mek: Before pWAs we had chrome apps and they did support non-square, transparent windows. But nobody complained when it was deprecated
cabanier: If you use effects on the page like background blur, would you expect those to interact with things behind the browser? Conceptually yes but would that work
adekker: Probably no. I like the explainer because you're hinting to the browser what materials to use
diekus: We wanted to create the windows effect or the MacOS effect and see if it's that possible
diekus: Would need to abstract the concept because when these OS's change to a new design language, instead of the actual name of the material you would have material 1, material 2...
cabanier: would be hard to make this a standard
Brandel: Would get challenged about how the intent of the materials get mapped. Nobody except the people who can make our materials run performantly have the ability to tweak these materials
cabanier: browsers are free to change colors for accessibility reasons like high contrast
cabanier: Seems like you would want something like backdrop blur on the background instead of just transparency
adekker: Having a set of materials where one is transparent, one is blur, defer to the OS to handle performance
Siyaman: would be useful to have transparency and also delegate to OS. Looking at different OS's they have different materials. Would be useful to have some materials you can use side-by-side in the same document
cabanier: How are you going to design your webpage for something you don't even know how it will look
Siyaman: yeah there could be light vs dark material
Mek: there's media query for light vs dark
Brandel: AVP has html model element with realityKit. Composited by core automation to be able to do those things. Intrinsically when dealing with 3d elements you're dealing with a 3d engine not 2d compositor of chromium. Maybe interesting to think about allowing spatial models or ensembles to exist in their own right. If intent of transparent
window is to convey more 3d dimensionality instead of that being the primary intent, gives us a different set of renderer constraints to lead toward
baranu: So could you use transparency for just a hole punch
Brandel: Sure or you could pull models out of the page entirely
cabanier: Would CSS still apply if you pull things out of the page
diekus: Remember back in the day when Nvidia 3d vision was a thing. In Microsoft IE there was a website of a ski resort with a model of a mountain, Nvidia was doing something to bring the 3d model outside, mountain spinning in front of you. Not sure how they did that. Pretty cool experience. Still confined in a frame, normal HTML content, somehow
the browser was detecting the model and pulling out of page. Rest of DOM content was flat
adekker: What I'm thinking about now, maybe pWAs seems to be the obvious next step
diekus: Starting small, in your images what I see are a bunch of small applications floating in the area
diekus: If we wanted a standard solution for materials maybe this would be a CSS thing. If we wanted standard materials could just start with opaque, translucent low, translucent high
diekus: At the moment we can't content web content on a translucent frame, first problem to solve
diekus: If you get a frame on windows or visionos or meta quest, and get transparency, if I define background color transparent is that even possible? Bad for performance? Start simple, get a translucent rectangle
Siyaman: I'm from bytedance, on visionos and quest(??) you can do this, you can make webview and background of window transparent, works quite well. You can play around with different translucencies
diekus: If this is supported could make this a hint to web developers
adekker: Report to the web developer as a CSS media query if it worked
cabanier: so just PWAs because it's easier to add to manifest?
baranu: is clickjacking a real problem, has come up as hypothetical issue
cabanier: Only an issue for fully transparent, not background blur. If something is 100% transparent maybe you draw an outline. Let the OS handle it
adekker: for click jacking concerns it's on the OS to handle it if provides a transparent material
cabanier: would it help if you start by constraining to the size. E.g. if weather widget / calculator, to start with, constraint to small dimensions?
adekker: Have you thought about security concerns diekus?
diekus: If you have something using micah you still have the frame, trusted UI. In explainer what we're looking at wasn't non-rectangular windows, just a normal PWA
adekker: This was helpful. the explainer is a decent start
diekus: Let's keep the conversation going, this is a first very early brainstorming idea
diekus: Happy to see there's interest beyond windows & mac, very cool to see for spatial devices makes even more sense than on desktop platforms
Siyaman: We're very interested in this, maybe we can work together. Did you also consider panels within the window, let's say there's a div within the page that wants to have different translucency
diekus: As long as there's micah / ultra thin / thick transparent frame, everything else is handled with CSS e.g. if I wanted to do some extra blurring. Challenge is how you can get web content with transparent background
Siyaman: Can you have micah on top of micah?
diekus: No you would just handle that via CSS
cabanier: Maybe we should start a community group for this, repo, monthly meetings
Mek: Just use WICG
diekus: Define what the use case is, get feedback from bytedance, meta, apple. E.g. we need new CSS or web app manifest or both
diekus: Next step is to go incubating. Fact that you three vendors are interested is a good start
adekker: Anyone look at this going you're crazy?
adekker: Apart from performance, we know that'll be an issue
adekker: We'll go off these next steps, I'll sync with diego
Siyaman: let's connect