16:55:27 RRSAgent has joined #immersive-web 16:55:31 logging to https://www.w3.org/2025/03/18-immersive-web-irc 17:00:45 lgombos has joined #immersive-web 17:05:15 Sorry people on IRC, the camera is still being set up 17:07:01 trevorPicoXR has joined #immersive-web 17:12:30 m-alkalbani has joined #immersive-web 17:15:01 lgombos has joined #immersive-web 17:23:20 present+ Laszlo_Gombos 17:23:31 atsushi has joined #immersive-web 17:24:42 rrsagent, make log public 17:24:46 Brandel has joined #immersive-web 17:24:54 present+ 17:25:02 mkeblx has joined #immersive-web 17:29:48 present+ 17:32:37 Sorry still having audio issues from the room. 17:34:45 present+ 17:34:55 bajones has joined #Immersive-Web 17:35:30 bajones has changed the topic to: https://github.com/immersive-web/administrivia/blob/main/F2F-March-2025/schedule.md 17:35:40 present+ 17:36:17 present+ 17:36:19 trevorPicoXR has joined #immersive-web 17:36:44 Lucas has joined #immersive-web 17:37:00 present+ 17:37:24 present+ 17:37:40 Emerson has joined #immersive-web 17:38:16 Ali has joined #immersive-web 17:38:21 present+ 17:38:25 jinch has joined #immersive-web 17:38:34 present+ 17:39:45 ada: explains the minutes 17:40:09 q+ to demonstrate how to ask a question 17:40:21 ack ada 17:40:21 ada, you wanted to demonstrate how to ask a question 17:40:23 q+ 17:40:26 q- 17:44:54 no audio on the VC if anyone's speaking 17:44:57 yonet has joined #immersive-web 17:45:04 present+ 17:46:48 atsushi has joined #immersive-web 17:47:34 rrsagent, publish minutes 17:47:36 I have made the request to generate https://www.w3.org/2025/03/18-immersive-web-minutes.html atsushi 17:48:14 agenda: https://github.com/immersive-web/administrivia/blob/main/F2F-March-2025/schedule.md 17:48:37 meeting: Immersive Web 2025 March face-to-face Day 1 17:48:40 chair: Ada 17:48:51 topic: Geospatial CG Introduction 17:49:08 scribenick: trevorPicoXR 17:51:23 MikeW has joined #immersive-web 17:56:05 present+ 18:03:43 q+ 18:03:43 [demo] 18:03:52 q+ 18:04:09 ack bajones 18:04:21 q++ bajones 18:04:27 q- + 18:04:41 room is not audible at all on VC 18:05:00 Working on that, sorry. 18:05:02 ali to do demo 18:05:38 I think the audio only worked because the presenter was not muted (i.e. it wasn't the room's mic that was being used) 18:06:00 Yeah, I just wanted to check if the room's mic had come back. 18:06:27 Sorry, I can't see the calendar invite. Could I get a link to the zoom meeting? 18:07:00 Sorry we can't paste it into IRC please log in to w3.org and click on your profile to access the calendar and the links 18:07:18 ali: open ar cloud is creating a way to anchor content to real world locations 18:10:15 ali: spatial clients provide a way to access location information to position virtual content in the world. 18:10:45 ali: shows a way to map our current location to visualize points of interest around our area 18:11:26 ali: once you are localized you can view GLB content or WebXR content 18:13:07 q+ to ask which features need to be enabled 18:17:19 ack bajones 18:17:47 brandon: great presentation, interesting demo 18:18:38 brandon: drilling down into logistical problems, big thing is coordinates on a global scale, you run into floating point issues, on web we use 64 bit doubles, not sure if that is sufficient 18:19:24 brandon: services process these numbers as 32 bit floats, this might not be enough precision to get a smooth range of motion. Would like to know how that can be solved. 18:20:11 mikes: we have a test that is working with normal numbers in javascript, it is working and we are targetting high precision. 18:20:44 brandon: to clerify, is all you calculations being done with number type and it has been sufficient? 18:21:09 q+ 18:21:10 mikes: we had doubts ourselves but it seems we can hit 10 micron accuracy, only see issues when you get very close 18:23:11 brandon: good to know, we might run into some issues with webXR as it exists today. Parts of the spec returns matricies that return lower precision. Other parts of the spec use array of number. Some parts of the api have high accuracy but others have worse so there might be some issues here. We might have some 2 tier solution where we pick one spot 18:23:11 in geo space but do local math using local space. 18:23:47 ack cabanier 18:23:50 mikes: current hardware, you receive geo pose with entity and you transform it into local coordinate space. This is how it works. Would like to accelerate this with improvements of api here 18:24:25 Ali's presentation link: https://www.openarcloud.org/oscp/w3c 18:24:33 rik: it seems this could be a specialization of the unbounded ref space. Get a geo ref space which gives you a patch of space and a position within that space. This could solve the accuracy problem 18:25:23 mikes: sphere has a limit when you move a far distance, you end up in a different position, need to be aware that if you create a geopose with more than 90 deg of latitude, you are on the other side of the world 18:25:49 rik: this is handled by the reset event, having poses in different geo spaces should just work 18:26:18 mikes: think about this as being able to generate local spaces accross the world. Having a connection to a satalite or tracking a space ship you can do so by thinking in terms of the earth 18:26:31 rik: if you create a ref space in the geo pose, you might need to add param to say which geo pose 18:26:46 mikes: yes there are many coordinate systems and want to limit as much as possible 18:26:54 q+ 18:26:55 rik: thinks this sounds possible 18:27:10 mikes: we are working on translation tools to convert other coodinate spaces to others 18:27:23 rik: lets standardize on one so we dont run into endless types 18:27:35 ack alcooper 18:27:35 alcooper, you wanted to ask which features need to be enabled 18:27:50 alcooper: tech question, you use webXR incubation features, which ones did you neeed? 18:27:57 ali: not aware of which 18:28:11 mikes: it was needed a while ago, now im not sure 18:28:24 ali: will follow up on this 18:28:54 alcooper: only spec not enabled by default is layers spec, you might not actually be using these, all camera access was shipped a while ago 18:29:09 mikes: we didnt have camera properties, had to create own set of metadata to store this info 18:29:10 ack Emerson 18:30:10 Ali has joined #immersive-web 18:30:50 Emerson: Precision of even a small area, if you scan a parking area and someone did something similar 200 m away. When you do slam stuff you do a bundle adjustment and have to keep doing it. The demo might work in a certain area, does it work if you are moving across distances and have many entities. Even in a small scale is there something that 18:30:50 needs to be solved to be continuous. What are your thoughts. Does mm precision does it scale to moving far distances 18:31:58 mikes: when i say mm i mean for interaction, when it comes to movement around large space, delegating this issue to other systems. I dont know how to answer this directly. 18:32:56 nazih: when you scan some area, there is accuracy and the further you go there will be less acuracy. Other systems like street view can handle this for accuracy. 18:33:42 ali: in a dream world, if google anchors are geopose compliant, this will be a better way instead of users scanning directly. AI can handle position correcting in the background and privide more accuracy 18:34:08 ack bajones 18:34:26 jinch has joined #immersive-web 18:36:01 q+ 18:36:02 brandon: the problem can be hit with just something in the same room. If you walk around a table, if you just use the single ref space, you might see some drift over time. Anchors exist to solve this problem, they have local points processed internally in the device, it uses landmarks to update the pose. For geopose you can get info from the cloud, 18:36:02 the global position (using lat long, or something for global). Then unbounded space is aroud that to get the intiail position. Then you create an anchor which is used to manage local update for better accuracy. 18:38:49 brandon: i am curious about getting the initial lat/long, gps alone is not sufficient. Using visual position systems was talked about. We dont want to build a perception that you need to ping a big companies server to get location. If it needs to rely on a visual positioning system or similar, the question of where that comes from becomes a 18:38:49 concern. If there was something well understood, we can point people to use some service. If we want to build this into the standard, we need to describe in concise terms in a way that it is not dependent on a single known service. It is fuzzy how we can do that without jumping out to an external server. 18:39:02 brandon: what do you perceive as the process for that? 18:39:40 mikes: geopose standard is what we are working on. Basic unit is pos/location in quaternion. This is agnostic of how you obtain it. 18:40:08 ali: it can be a geoposed qr code, doesnt matter how these poses are obtained 18:40:46 ali: would like to set the standard of how to obtain/use it, doesnt matter where it comes from 18:41:51 emmerson: the spec could have something like, if you use google or other service, they specify the precision. 18:42:09 mikes: this issue isnt solved but we have specific formats to deal with that 18:43:38 brandon: from a users perspective, there needs to be a well understood flow of how this info gets connected. I can see this encoded into a qr code, if that was common, we could tell people this is how people can use the api. People in this room might think this is a good usecase. The spec might say we dont worry about how the user gets this 18:43:38 information. 18:44:27 brandon: maintaining services like this by browser developers can be expensive and a complex/political process to setup 18:45:17 brandon: we could say at the api we dont deal with where the location comes from, the user of the application can choose where to pull from (specific service or qr code) that associates that a specific lat/long is related to a certain ref space 18:46:35 brandon: this could make this service stuff external to the api otherwise the browser weill have a location service that they have to pick a location provider from 18:47:57 mikes: we have the need to operate with geospatial information that is encoded in a specific way, we dont have a camera that is geospatial, how do we create a camera that can integrate that data. We have a camera, with lat / long, rotation, north. that is everything we need. 18:47:58 ack cabanier 18:48:34 rik: would be better if every geo ref space could come with its own anchor you can create. The more anchors you have the slower things become 18:48:55 mikes: those anchors are also entities in the geospatial element. There is no origin point other that north pole 18:49:09 rik: anchor is just a point in space, can define that as the center of your geo ref space 18:49:46 mikes: you are always in a ref relative to earth. The frame of ref is always the center of the earth. Those anchors are the same level as anything else you located 18:51:17 brandon: when you manage to geo locate youself, you wil let geocoordinate that is preceived by the device. it would be convenient for developer to be able to drop an anchor at the time of location where the device itself has a series of landmarks so the anchor is always relative to those. It has nothing to do with the global pose. If we have an 18:51:17 anchor at the time as global pose. The device can position the anchor always tied to that specific geolocation. 18:53:19 mikes: want to generate points as a go, it is the responsibility of those that are working on the geolocation api. While it is important to have a good geolocation api. it doesnt mean that the idea we are proposing. If the device knows where it is, i dont mind if other entities exist. 18:54:36 ali: if you are in your home you dont need geo pose. But that can also know everything around it, if it is tied to a anchor you can place them relative to other anchors. At some point this should be at device level where you pick which geo service you want at the device level. 18:54:43 q+ 18:54:46 ack bajones 18:54:53 q+ to ask about lunch 18:55:43 brandon: what you are showing so far relies on visual positioning systems. Do you a have a sense of what information is needed to get an accurate geo location. Is it a single image, series of image, point cloud etc? 18:56:15 ali: you just send one image along with lat long, uses gps to narrow down its map search to match it 18:56:26 brandon: cool that it comes from a single image 18:56:35 ali: the area has to be scanned earlier 18:57:13 ali: there were 21 images to scan the area at first and this gave accurate geopose at the service provider level to generate things before hand. 18:57:25 brandon: ok, so user creates one picture. 18:58:39 ali: you can see where you should be to localize. Its not everywhere, there is a predefiend location where you start and take the image. Things will be easier with ai. At the backend there can be localization done with anchors. 19:00:43 brandon: if the api looks like webxr delivers locally image taken from camera that we can take the image and metadata, the cloud can figure out where the picture was taken, i can figure out where to place the anchor and start building out from there. This could allow for system that isnt tied to a specific. THe api is a little odd but we talked 19:00:43 about something similar with qr codes. This might not be right but i could see it working 19:00:55 ali: agree 19:01:17 mikes: great for a geolocation api but its separate from webXR, if we only have 5 mm accuracy its not great but we can work with that 19:01:47 lunch time 19:02:49 Ali has joined #immersive-web 19:02:54 Here is the link for my slides 19:02:54 https://www.dropbox.com/scl/fi/rps3p43rarclvkdx9gvh0/GeoPose-SWG-W3C-Immersive-Web-Face-to-Face-March-2025.pdf?rlkey=w7a45h01vuo0tziipivu8dp2v&dl=0 19:02:54 and 19:02:54 the web site URL for today's demo is below 19:02:56 https://www.openarcloud.org/oscp/w3c 19:03:04 rrsagent, publish minutes 19:03:05 I have made the request to generate https://www.w3.org/2025/03/18-immersive-web-minutes.html atsushi 19:03:46 topic: [lunch break] 20:10:26 Sorry we're still running late 20:11:34 trevorPicoXR has joined #immersive-web 20:13:55 yonet has joined #immersive-web 20:14:22 Starting back up. Brandon is discussing in person logistics. We will have the current setup for the rest of the day 20:15:30 Brandel has joined #immersive-web 20:15:36 bajones has joined #Immersive-Web 20:15:43 present+ 20:15:47 jinch has joined #immersive-web 20:15:52 present+ 20:15:56 present+ 20:16:18 scribeNick: Emerson 20:17:06 ada is explaining the next steps 20:18:59 item: https://github.com/immersive-web/webxr/issues/1402 20:19:19 ada: issue relates to AR passthrough features 20:19:47 ada: should we have a new session type 20:20:56 cabanier: how do we expose it, maybe set it in the render states 20:21:13 q+ 20:21:19 q- 20:21:21 q+ 20:21:28 ... open to any other ideas 20:21:47 ack bajones 20:22:13 bajones: Render state is probably the right spot for it 20:22:17 m-alkalbani has joined #immersive-web 20:22:18 Lucas has joined #immersive-web 20:23:31 MikelSalazar has joined #immersive-web 20:23:46 bajones: there is convoluted way to get by without introducing a new api, but it's a pain 20:23:53 q+ to discuss what else is meant by being "in AR" as a bundle of permissions 20:24:09 ack mkeblx 20:24:38 MikelSalazar: do we care if its accurate at one frame 20:25:26 cabanier: we would only disable the second stage, it would come up right away 20:25:46 bajones: yes, everybody can guaratee the type described in the spec 20:26:13 ... these days its mostly VIsionOS, OpenXR and some other solutions 20:26:24 q+ 20:26:48 bajones: bigger question I would have is it a hint or is it a guarantee 20:27:30 ... video passthrough devices might end up implyng you have to do another clear or copy if a device dosen't have the ability to stop the pass through 20:27:43 I don't need the pass through right now, you're allowed to disable it 20:27:49 I feel the naming is important 20:27:55 ack Brandel 20:27:55 Brandel, you wanted to discuss what else is meant by being "in AR" as a bundle of permissions 20:28:09 Brandel: makes sense as a method of transition between AR and VR 20:28:22 q+ to mention directionality 20:28:28 .. what is understood to be meant between AR and VR session for a user 20:29:00 q+ 20:29:14 .. it would be worth while to look at lower level switch bases on user persmisison 20:29:36 alcooper: on mobile would be something you are already in AR, 20:29:36 atsushi has joined #immersive-web 20:29:56 Brandel: user notification is important 20:29:56 ack cabanier 20:30:01 q- 20:30:12 cabanier: could just render black over pass through 20:30:12 rrsagent, publish minutes 20:30:13 I have made the request to generate https://www.w3.org/2025/03/18-immersive-web-minutes.html atsushi 20:30:40 s/item: /topic: Hint for VR in AR WebXR sessions / 20:30:43 .. already start in AR , user is already passed the permission 20:31:15 alcooper: I'm not sure about enviroment blend modes 20:31:38 .. Android XR don't know 100% how it works 20:31:44 ack bajones 20:32:25 q+ 20:32:27 bajones: we do in some cases shuffle the camera image around 20:32:51 .. agree with Rik, in terms of premissions, we want to deal with at the session level 20:33:37 ... in AR session, once you've done that , its not realy a mode change, you don't need to spend time processing these pixels 20:33:46 q+ 20:34:14 I'm not revoking persmission by saying I don't want to share camera , optical passthough is a good analogy 20:34:24 q+ is this similar at all to depth sensing feature that already exists? 20:34:39 Brandel: its a matter of duration 20:34:42 q+ similar to depth sensing? 20:35:22 bajones: I can get on a zoom call, meeting goes muted, I walk off and my camera could still be runing, to certain degree that is on my as the user 20:35:54 there is only so much we can do to protect users from themselves 20:36:06 the most practical thing would be to have some sort of Indicator 20:36:19 ack ada 20:36:23 .. like if I bring up a task bar, highlight it 20:38:08 ada: on a s diff topic . we don't explicitly tie features to session types. there are certain features only excted to work if explicitly saying where you can acessess them. 20:38:10 ack alcooper 20:38:21 alcooper: I somewhat disagree a bit 20:38:42 there could be valid scenarios for developers to do that 20:38:58 on chrome we have VR and AR separte persmissions 20:39:23 It should be up to the user agent if its ok to grant persmissions based on "persmission model" of ua 20:39:45 ..without raw camera , sites don't have access to raw camera that is part of the passthrough 20:40:08 q+ trevorPicoXR 20:40:09 .. the privacy concerns are a little redundant as the site can do that anyways 20:40:14 q- similar 20:40:44 ada: would like to clarify, assumption we disable all AR features, but if this is just for redering, than ok 20:41:09 ack trevorPicoXR 20:41:10 cabanier: the only thing that dosent work is for unbounded spaces 20:41:26 q+ 20:41:34 trevorPicoXR: whatever we sovlve for this may apply to other features as well 20:41:40 ack ada 20:42:01 ada: maybe you could't do unbounded , would you want to have a feature for this 20:42:31 if the users can know if they require an unbouned space they should not if its not going to work 20:42:58 cabanier: even if its ubounded ther eis not much we can do 20:43:08 unbounded has nothign to do with pass through 20:43:23 if you are in unbounded you don't get the guardian 20:43:25 q+ 20:43:30 ack alcooper 20:43:30 ada: probably ok 20:43:45 ruoya has joined #immersive-web 20:43:49 alcooper: per Brandon's point, indicate it's a hint not a guaranteee 20:44:01 there are form factors that simply can not do it 20:44:16 ada: what is the current proposal name 20:45:01 {uShallNotPassThrough} 20:45:15 {passThroughObscured} 20:45:25 bajones: pass though obscured 20:45:46 jinch:pass through limited 20:45:59 {dontNeedPassThrough} 20:46:37 bajones: there might be some devices can't do it 20:46:52 framing it positively as {needsPassthrough} 20:46:54 ..we want something that indicates the state of the app 20:47:23 .. if its a heavy lift we are giving premission to turn it off 20:47:33 ada: could we mandate to clear to black 20:47:47 canDisablePassthrough(Compositing) 20:47:55 bajones: we may not wish to mandate 20:47:55 suggested passtrhoughAllowed" 20:48:11 MikelSalazar: can disable passthrough 20:48:41 bajones: we want default to be falsy 20:49:05 .. having something where we are actively saying allowed disabled is probably the better framing 20:49:17 ada: I like passthrougobscured 20:49:29 isPassthroughObscured? 20:49:54 isPassthroughFullyObscured? 20:49:55 alcooper: passtough objscured vs fully obscured 20:50:15 passThroughObscured 20:50:31 passThroughFullyObscured 20:51:07 -1 20:51:07 ada: voting on the above 20:51:09 +1 20:51:10 +1 20:51:11 +1 20:51:12 Fully = +1 no Fully = -1 20:51:12 +1 20:51:12 -1 20:51:13 +1 20:51:13 +1 20:51:14 +1 20:51:19 +1 20:51:27 -1 20:51:33 +1 20:51:52 mmerch has joined #immersive-web 20:52:18 ada: Pass Through Fully Obscured wins atm 20:52:18 pasthroughFullyObscured passed 20:52:57 ada: we can wrap up that issue 20:53:09 To be a bit pedantic obscured is to make it dark, 20:53:29 occluded is to hide it behind something else 20:54:05 Also, here is the link to the slides of the previous presentation: https://docs.google.com/presentation/d/1ozukf7t8WPiZn9p74gn16JNJuwKNiR07UTs0Pd1M_AU/edit?usp=sharing 21:03:29 topic: Depth Sensing: Raw vs Smooth Depth 21:03:32 topic: https://github.com/immersive-web/depth-sensing/issues/51 21:03:46 scribeNick:Brandel 21:04:11 chair: yonet 21:04:12 alcooper: these next few issues are related. 51 + 52 made us come up with 53 and broader changes to the API shape. 21:04:53 ... ARCore and OpenXR, things we use, support "raw" and "smoothed" depth information. 21:05:25 ... Currently, we don't differentiate or inform about the characteristics of the depth texture. The PR I'm working on adds a preference for if you want raw or smooth 21:05:46 ... It would be a preference, such that if a UA has only the ability to offer one it can do so. 21:06:11 q+ 21:06:20 ... As I started going back through the spec to re-work three things instead of two, I ran into some things that made me want to refactor. 21:06:25 ack cabanier 21:06:39 m-alkalbani has joined #immersive-web 21:06:47 cabanier: We don't have support for different types - ours is only raw and we advise authors to do smoothing of their own. 21:07:32 cabanier: We _could_ add an extra pass? 21:08:01 alcooper: We would have to make it optional so that it supports environments where only one can be offered. 21:08:17 cabanier: We should also report if a hint is denied, when you ask and don't get the type 21:08:40 alcooper: I think if you ask and don't get it, it should be able to reject - the main question is probably in a retro-fit situation 21:09:07 cabanier: I would like it so that if you ask for smooth and the session reflects raw, that's a good signal that the author will have to do the smoothing themselves 21:09:18 alcooper: so you'd prefer if it _never_ rejects? 21:09:30 cabanier: I think so, does that seem reasonable? 21:10:12 alcooper: That does make sense - it probably breaks less in the event that people have stuff that works today that might break if a "smooth" request is now denied on Quest 21:10:25 ruoya has joined #immersive-web 21:10:41 cabanier: If we get feedback from developers indicating that they _don't_ want to do this work themselves, we could add one. It seems like what we'd do is more or less equivalent to what they'd do. 21:11:18 alcooper: There's a possibility that a UA is in a better position (either by way of additional data etc) to be able to do a better job of this, but doesn't have to be a hard guarantee 21:11:30 topic: https://github.com/immersive-web/depth-sensing/issues/52 21:11:33 alcooper: On 52 - Trever helped foreshadow this one 21:12:14 ... Should we be able to pause or resume sending up depth data, based on the expected cost? Maybe use snapshotted data etc once an object has been placed? 21:12:31 q+ 21:12:37 ... Pausing could be void and resume might need to be a promise, since it could take a few frames to resume a service 21:12:46 ack cabanier 21:13:00 cabanier: Depth takes a couple frames for us to get back to us - but we might serve frames that happen to be empty 21:13:12 q+ 21:13:14 alcooper: I think we return null rather than empty data until we get meaningful things 21:13:49 yonet: In the case that you need to do extra scanning, what would you do? 21:14:06 cabanier: We would wait the few frames like we currently have to at the outset of a session 21:14:33 ack bajones 21:14:58 bajones: Nit: I'm not sure that this belongs in "render state", since it's not a feature of what is visually displayed 21:15:22 ... Render state tends to have guarantees about when things are going to take effect. It's not the end of the world to put this information in there. 21:15:34 ... The depth data comes in on the frame? [It's queried on the frame] 21:15:40 q+ 21:16:10 alcooper: Rik has a PR to get it in on an additional view. Right now it's _aligned_ with the frame's view and the PR is to make it come from a different one. 21:16:32 alcooper: I _think_ they come through the WebGPU binding, at least for the GPU one. The CPU one I'm not sure 21:17:06 bajones: In terms of where we set this, my initial instinct is to put it on the binding [cabanier shakes head vigorously] 21:17:32 alcooper: The XRFrame carries CPU depth info, XRBinding carries GPU depth info 21:17:42 ... I'm not sure if cabanier's change impacts that 21:18:01 bajones: My gut tells me the session is a better home for this information 21:18:32 ... that also helps preserve the function of the renderstate to provide guarantees about timing information about this 21:18:51 ack trevorPicoXR 21:18:53 cabanier: I'm fine with this not being in renderstate, I don't think it needs to have functions or fulfill a promise 21:19:16 trevorPicoXR: I like the idea of this being on the session, I'm curious about whether there's a path to follow this pattern for other features as well 21:19:53 ... we might want to (for example) disable Layers, or hit testing - we'd need to enable the most at the outset, but potentially pausing features and resume them 21:20:20 ... Is the challenge here that the API is complicated, or that we want to _make_ this simple? 21:20:25 q+ 21:20:30 ack alcooper 21:20:34 cabanier: The existing API is supposed to be able to accept null textures and dummy data 21:21:08 alcooper: What trevorPicoXR is suggesting is a more comprehensive model of being able to toggle settings at a more granular level. I think a promise would complicate that 21:21:35 ... But anything that's null that's not _expected_ to be null could complicate that and delay things 21:21:54 cabanier: I'm conscious that the slate of capabilities present slightly differently, which makes a unified approach more challenging to resolve 21:22:24 bajones: We can resolve some of that at the API level so that it's more of a hint and the user-space code isn't significantly affected 21:22:42 cabanier: Some of the things that trevorPicoXR suggested, though, aren't session-level features 21:23:16 q+ 21:23:16 bajones: someone gave an example of toggling on hit-test. For things like "requestHitTestForTransientInput" 21:23:37 ack alcooper 21:23:41 you can cancel that in order to lighten the burden, and it does currently live on the session 21:24:01 alcooper: Lighting estimation looks similar - you ask for a light probe and could probably just drop it after you don't need it 21:24:50 ... 53 gets toward a more unified approach that makes use of a tracker, and you can just destroy this object to indicate a disinterest in using those capabilities on an ongoing basis 21:25:09 topic: https://github.com/immersive-web/depth-sensing/issues/53 21:25:41 s/requestHitTestForTransientInput/requestHitTestSourceForTransientInput 21:26:28 alcooper: Should we reconsider the API shape of depth? As I was trying to implement this, I was wondering if a site might want both raw _and_ smooth, for example. 21:27:02 ... We also have other constructs that have a requirement like this, like a hit test, that you _indicate_ at start up but don't necessarily consume immediately or at all times. 21:27:22 ... I have a straw proposal that has an XRDepthConfiguration that indicates some preferences 21:28:02 ... If we're already saying that we need to start and stop things, or ask for both kinds of things at the same time, then maybe more things will follow in that pattern as we look harder 21:28:32 ...I'm not aware of other features that currently require an `init` - maybe DOMOverlay 21:28:54 cabanier: So in your proposal, you could have _multiple_ depth configurations at the same time? 21:28:57 q+ 21:29:05 ack bialpio 21:29:11 alcooper: at most one raw, one smooth - predicated and ordered on the depth type 21:29:41 cabanier: Depth is currently our most expensive capability, so I don't know that it's a good idea to make it easy to request something so hard/expensive 21:31:22 q+ to mention tighter preferences 21:31:26 bialpio: I think a subscription-based model might be more sensible here, since there are smooth/raw, cpu/gpu - not necessarily things that an author can pre-declare and express ahead of time 21:31:53 ... It might be easier to use a promise-based model where you can engage in an ongoing negotiation about whether you get A, and or then B, then C etc. 21:32:21 ... Rather than a single config that doesn't necessarily give a collection of options that the author has the ability to immediately respond with 21:33:07 ... I also wanted to respond to cabanier's thoughts on multiple sources of the same broad type - a system could reject things if it's at capacity. Sites would simply have to start tolerating more failure cases 21:33:42 q+ 21:33:43 ... It might become harder to write an app against disparate implementations, but maybe that's okay? 21:34:15 ack alcooper 21:34:15 alcooper, you wanted to mention tighter preferences 21:35:07 alcooper: one thing bialpio mentioned that contributed to my motivations on this, in addition the raw/smooth, is the idea that an author says "If I get raw data, I need it on the GPU, whereas I may be able to cope with smoothed on CPU etc." 21:35:36 ack cabanier 21:35:43 ... letting developers specify a tighter set, this configuration gives them more control to make a more precise request of things 21:36:25 cabanier: I am apprehensive that if there's too much variability about how this kind of session-granting can establish, it will make it too complicated for developers to anticipate how different devices will run 21:36:36 ... (and maybe we need to be a little more prescriptive) 21:37:11 alcooper: We do have a shipping API that we might be able to steer towards a more conflict-resistant approach 21:37:14 q+ 21:37:44 ... a new API allows us to build something that can better resolve some of the potential conflicts that we're currently tip-toeing around 21:37:44 q+ 21:38:20 ack bajones 21:38:34 cabanier: I think even where there are big disparities, we _can_ resolve them by doing things like dropping GPU buffers into something CPU-readable 21:39:37 bajones: I don't love when APIs construct elaborate preference sets in order to get around variability. I'm aware WebBluetooth that has something like this, as does aspects of video selection 21:40:00 q+ 21:40:10 ... It's my impression that this is pretty daunting to authors - in general we have tended toward a more prescriptive model that _tells_ authors what to expect 21:40:55 ack bialpio 21:40:59 ... Leaning more heavily on one or two primary views etc - and providing _support_ for edge (or CAVE) cases, but making sure that the most common path is going to be the best-catered to 21:41:43 bialpio: The only problem I have with mandating a specific course of action for all classes of devices, is that we might be effectively penalizing one of them if it happens to be very expensive to do that thing 21:42:24 ... if a category of devices simply doesn't _get_ data in a form that's amenable to the 'expected transformation' 21:42:52 ... this is a lot like the course that cabanier was trying to mitigate with re-projection of depth data, for example 21:43:45 q+ 21:43:55 ack alcooper 21:44:01 ... it's not a reason not to prescribe things, but we need to be conscious about the implications of saying "this is the expected presentation for a given source of data", and jeopardizing an API shape because it ended up hard 21:45:11 alcooper: Agree with bialpio. We have these flexibilities because different devices do things differently - it feels reasonable to say "if you ask for depth, you will get it in this way. You can _request_ if you want it CPU/GPU, or Smooth/raw" - sometimes those things are cheap and we should be able to send it to an author 21:45:28 ack cabanier 21:46:29 cabanier: I think we've learned a lot since depth V1. I want to reiterate that if we give authors too many options they will get confused. We should be able to make an API that says "this is the best way to work with this data" - and provide hints for what options exist 21:47:25 alcooper: We are using CPU data, it wouldn't be hard for us to upload that to GPU but it would be expensive to pull it back down 21:47:56 bajones: when you're in the WebGL then you can pass things around. The bigger challenge is when you have a bubble in the data that takes time to resolve 21:48:27 ... If you're willing to make the data asynchronous then the performance could probably benefit 21:49:15 alcooper: Back to my earlier concerns - yes it would be better to say there's one right way to do this, but I don't know that our devices are at a place where we can refactor all of these devices streams to behave the same 21:49:33 ... We might want to wait a few years to converge 21:49:42 cabanier: I don't know that we _will_ converge though 21:50:04 trevorPicoXR: this is kind of like camera data that can, at times, go straight to a GPU while some have to go through CPU 21:50:32 ... but a developer can be pretty indignant about having that complextiy masked, if it has impacts on the behavior and perf they can expect 21:50:59 alcooper: If we can make minor changes that are no worse than the current world then that would be good. We could make something less "opt-in" 21:51:38 q+ 21:51:53 cabanier: You might not like my proposal - what if we just drop CPU? Is anything going to use CPU directly, or are they just jam it in the GPU? 21:52:41 alcooper: I have a developer who wants CPU, (but also they say that one of the first things they do is throw it in the GPU) 21:52:47 ack bialpio 21:53:12 q+ 21:53:21 bialpio: I think the use-cases for CPU are physics and collisions, if you want depth for occlusion it goes to GPU. 21:54:15 ... per bajones suggestion, I would lean toward dropping GPU because it's simply easier to throw things _into_ the GPU. That might put an upper bound on the resolution of the buffers that this is viable with though. 21:54:48 ... I don't have a good intuition of where people might use CPU data than physics - AI possibly 21:55:08 q 21:55:13 q+ 21:55:14 ... like pathfinding based on the environment, but I don't know how common that use case it 21:55:22 s/case it/case is 21:55:27 ack alcooper 21:55:50 alcooper: The notes from the developer said that "raw depth is best on CPU, for physics, smooth=occlusion=GPU." 21:56:06 q+ 21:56:15 ... we might be able collapse some of this to say that "raw goes on CPU, smoothed goes on GPU. I'm not sure if that helps anyone 21:56:19 cabanier [shakes head] 21:56:19 ack bajones 21:56:49 bajones: Something that came to mind is that AI is almost certainly going to go on the GPU. 21:56:50 q+ to something something webgpu 21:57:08 q- 21:57:18 ... for physics I can _see that_ being more convenient on the CPU, but we also have a lot more compute-based abilities coming with WebGPU. 21:57:52 ... bialpio is right that it's almost always easier to get data _on_ the GPU rather than off it, but am conscious that these resources are going to continue to get bigger 21:58:35 ... There is an overlap in that use-case with things like environment meshing and awareness features. It seems like that might be preferable to depth in that event, though. 21:59:02 ... Depth might allow you to expand, resolve and refine a mesh, but it won't let you consider things behind you, for example. 21:59:20 ... I do think that convergence on being able to use one thing for all kinds of tasks is multiple years down the line, though. 21:59:44 ack bialpio 21:59:50 ... In the long term, I do think that GPU-based usage is going to supersede CPU-based approaches 22:00:15 bialpio: When I said "AI" I didn't mean ML, I meant _fun_ AI like A* and minimax etc 22:00:55 q+ 22:01:09 ... given that we have a pretty even split about use-cases - makes me think we should just offer an occlusion feature itself, rather than furnish the app with the data and expecting them to do it themselves 22:01:29 ack cabanier 22:01:33 ... that might serve to mitigate the need to do too many different things with this information 22:01:42 q+ 22:01:52 cabanier: It's not really feasible for us to composite in that way with today's technology unfortunately 22:02:25 ... most of the time you do occlusion in AR, so you only have to apply occlusion to a small character rather than the full overdraw of doing it on the whole buffer 22:03:01 zakim, pick a victim 22:03:01 Not knowing who is chairing or who scribed recently, I propose Lucas 22:03:08 ack bajones 22:03:16 scribenick: Lucas 22:03:23 Sadly, I am not fast enough with the keyboard :-/ 22:03:51 bajones: To support Rik on the Occlusion API, we can't just take the depth as provided and use it as a Depth Buffer 22:04:53 ... 1. It's not aligned and it's a lower resolution than want you want. 22:04:53 If you want to do high quality occlusions with these depth buffers, it's something you need to do as a render pass. Basically, Rik is right 22:05:37 ada: This is wild and might be terrible. What if you were to foveate the depth buffer. That way the depth would be cheaper to calculate 22:06:10 ... Trying to reduce the number of pixels 22:06:32 cabanier: Foveation cuts the frag cost, so this wouldn't apply 22:08:20 alex: It seems like we might move forward with depth v2. I don't know if there's enough to do away with the request model we have now. I think we may want to support GPU depth 22:09:09 ada: might want to take that back and think on it with company / clients 22:09:23 cabanier: we don't support cpu, so it's not a problem 22:10:07 alcook: Is depthv2 worth pursing? I think we should land the improvements we have in the mean time to give users a better experience sooner 22:10:27 q+ to boggle/reflect on the disparity of resolution on the sensors 22:10:44 cabanier: Maybe by the time v2 comes out, there will be a clear to not support cpu 22:10:57 alcooper: Maybe data format wise, it would be easy to keep them all 22:11:20 cabanier: Which formats do we support? 22:11:24 ack Brandel 22:11:24 Brandel, you wanted to boggle/reflect on the disparity of resolution on the sensors 22:11:25 alcooper: 3 22:11:58 Brandel: I didn't realize the resolution between the two approaches were so different. It makes sense that there's a difference in needs 22:12:35 alcooper: I think that's the crux of why we have cpu + gpu, then there's the element of having smooth and raw for the different purposes 22:13:13 Brandel: We should acknowledge the function of the depth API for both platforms 22:13:24 q+ 22:13:25 cabanier: I think we both want to use it for occlusions 22:13:53 ack bialpio 22:14:03 bajones: In the spec we have luminance-alpha, float 32, and unsigned short formats 22:14:40 bialpio: I think there were other resolutions that could be support on our platform, but it might require dedicated hardware. Still a bit away from 1k x 1k 22:15:13 ... The other thing here is that there may be something different on how the depth buffer is computed. I think ARCore is trying to do live scanning of the environment. 22:15:54 ... Feeds in a bunch of data... This might be the reason why the resolution is still pretty small 22:16:37 ... I think they don't want to assume what kind of device hardware is supported. It really depends on how things end up developing there. For now, it seems we're stuck with fairly low res on phones without dedicated hardware 22:17:20 alcooper: I need to think about some stuff here. I don't know how to proceed, but not sure if more discussion is beneficial for the time being 22:34:17 yonet has joined #immersive-web 22:35:47 topic: https://github.com/immersive-web/layers/issues/310 22:45:13 yonet has joined #immersive-web 22:48:20 Brandel has joined #immersive-web 22:48:27 present+ 22:51:12 ruoya has joined #immersive-web 22:51:45 jinch has joined #immersive-web 22:51:45 scribeNick: MikelSalazar 22:52:13 scribeNick: mkeblx 22:52:32 https://github.com/immersive-web/layers/issues/310 22:52:46 bajones: i don't think we came to a conclusion on api shape 22:53:34 cabanier: threejs is moving to a 2 pass model, so in order to apply foveation need to apply foveation to render texture, and no webgl extension to do so 22:53:59 cabanier: and webgl kind of on the way out, and slow to get an extension 22:54:10 q+ 22:54:20 ack bajones 22:54:27 cabanier: proposal: foveatedTexture with one parameter (bikeshed name) 22:54:32 rrsagent, this meeting spans midnight 22:55:11 bajones: is that the api shape that points at a texture, and say foveate that, and then whenever it is rendered to it is foveated? 22:55:23 cabanier: yes thats how would work 22:55:51 bajones: this would probably be applied on the webgl binding, would be different on webgpu backend 22:56:28 cabanier: you just say where the eyes are and the regions beyond that, but there's still hand wavey things 22:56:41 bajones: anything else that need to be set? 22:57:11 cabanier: it kind of has some weird attributes but that's similar to other layers, webgl layer 22:58:09 q+ to comment about post session state 22:58:45 bajones: is it the best path to have a texture once set as foveated will be foveated and can't turn off, if want off then recreate a new one w/o etc 22:59:01 q- 22:59:39 bajones: traditionally with webgl can usually query this state, do we need? i.e. 'is this texture foveated?' 22:59:39 cabanier: then would need to keep track of state, we are taking the easy way out w.r.t. 'proper' webgl way 23:00:27 ada: it would be weird if once left webxr experience, see the foveation artifacts potentially on non-webxr experience 23:00:45 q+ to ask about eye tracked foveation 23:00:54 ack bajones 23:00:54 bajones, you wanted to ask about eye tracked foveation 23:01:09 ada: maybe a set with textures that have foveation, so developers can do clean up on those textures / discard / re-create etc 23:01:42 bajones: this would be something that persists outside of the session so that could be a negative, 'leave campsite as you left it' is a good principle 23:02:39 bajones: tried to replace fixed foveation as this would be better, but there is a dependency on eye tracked foveation to get the biggest worthwhile benefit 23:03:14 bajones: q to apple forks: can we do in a way privacy preserving way? 23:03:35 ada: can't speak for apple in terms of if would be possible 23:04:23 Brandel: it's done at system level on visionOS, so would have to be in a way safari/webkit wouldn't know about it 23:05:33 Brandel: don't know if visionOS lets apps do dynamic (non-fixed) foveation 23:07:05 bajones: even if you don't know exact foveation texture etc you can figure out through various side ways aspects of the foveation etc 23:08:07 ada: others could speak more on what is really possible and would need that to really give fuller answers 23:08:36 Brandel: visionOS has a more complicated foveated rendering even for fixed 23:09:11 bajones: do you have any sense how you would apply foveation in the deeper layers, and how would do this from webkit? 23:09:26 Brandel: don't know 23:09:57 bajones: want to know just so that can make sure proposal is implementable on platform 23:10:08 ada: overall seems a good idea 23:10:18 q? 23:10:31 cabanier: so should state persist after session ended? 23:10:57 cabanier: texture is created as focus, but has foveation applied 23:12:02 ada: opening a potential trap or visual weirdness or closing potential options for headsets that do more unusual things with foveation 23:12:33 cabanier: answering if change foveation level if need to re-create: no 23:12:57 ada: don't like idea of device state leaking after session end 23:13:32 cabanier: could do something to unapply foveation, but feels ugly 23:14:30 bajones: but maybe worth it as it's cleanup and could be good to do from session being -> end, weird to have some hidden state changed and can't tell 23:14:44 bajones: could do some best effort thing to not leave it hanging out 23:15:01 cabanier: so would have to add that to the spec? 23:15:17 bajones: yes would have to add something related to session end 23:15:42 cabanier: could use a weak list, as we don't want to hold on to them, could have 1000s of textures 23:16:20 bajones: i'm sure other specs has weak lists, it exists but don't know details exactly 23:16:55 bajones: can't use weak list as not iterable 23:17:05 ada: can do with weak set 23:17:41 cabanier: ok something possible 23:18:03 cabanier: is the name foveateTexture alright with everyone? 23:18:47 bajones: likely for webgpu would have to do something different, likely would have to do something at beginning of render .. would have to look into; but for webgl this seems fine 23:18:50 q? 23:19:04 q+ for other hints about VRR 23:19:11 ack Brandel 23:19:11 Brandel, you wanted to discuss other hints about VRR 23:19:24 Brandel: foveation is not the only reason to have variable raster rate 23:19:46 Brandel: is that something we'd want to address? 23:20:46 bajones: correct that variable rate stuff is useful outside webxr, but 1) webgl working group not interesting in doing a lot of the work required for maintence mode webgl 23:21:27 bajones: for webgpu do want to consider variable rate and may overlapy with eye tracking, but that's farther off 23:21:54 bajones: for now like the more straightforward foveation being described here 23:22:59 bajones: typically variable rate has a generated texture map of shading rates and may eventually go that direction 23:23:41 bajones: I assume this would take a param of foveation level? 23:23:59 cabanier: no would take the layer's foveation level set, a float 23:24:13 bajones: how do we make the assocation between layer and texture? 23:24:36 cabanier: so i guess we would to add param 23:24:52 bajones: either we can pass as creation time or find some way to associate 23:25:29 cabanier: github issue proposal has the param there 23:25:49 bajones: any way to change so dont need bound texture for param 0, i.e. have unbound 23:25:53 q+ 23:25:58 ack ada 23:26:24 cabanier: would have to handle binding if didn't do that which is ugly 23:26:40 cabanier: want to avoid storing strong references for these textures 23:35:10 : react three fiber is nice has some ways to make UI a little easier in WebXR, but doesn't work with all HTML etc it's restricted and most companies don't want to have to rebuild, so we're building a platform somewhat similar to react native that lets developers build 3d applications easily across platforms in a mult-app way 23:35:52 q+ 23:36:10 ack bajones 23:36:27 : we want to come to all platforms like meta, androidXR, etc and want to collaborate; originally wanted to build upon webxr platform but needed native features and functionality 23:36:48 https://github.com/WICG/canvas-place-element 23:37:43 bajones: awesome great to see the experimentation. model tag is something that gets out of webxr fully immersive limitation, also wanted to point out placeElement api which will allow UI etc things to be much easier 23:38:35 bajones: Also there are possibilities of things like DOM layers that are possible that would be helpful for UI among other things 23:40:23 q+ to talk about the uniquely dangerous role of a browser 23:40:26 : If we don't use HTML are we losing the spirit of the web? 23:40:26 : We don't want users to have to rewrite their HTML etc, we do have concepts like a volumetric view 23:40:56 : I don't like that UIs are still so flat 23:42:18 : We can do 3D stuff, we can map HTML/DOM to native elements that may do 3D; open to exploring more but still early 23:43:23 : think of this like electron, that can talk to native stuff but still web-y; think of like that + combining with react native 23:44:01 jinch has joined #immersive-web 23:45:15 ack Brandel 23:45:15 Brandel, you wanted to talk about the uniquely dangerous role of a browser 23:45:44 Brandel: one reason browsers are slower is for security due to untrusted and need to check, so one way to combine things to get web-like benefits and concepts but avoid some of the security issues as trusted concept is an interesting concept, but eventually would like web itself to be able to do many of these things, cool that can experiment before 23:45:44 that exists 23:48:20 rrsagent, publish minutes 23:48:22 I have made the request to generate https://www.w3.org/2025/03/18-immersive-web-minutes.html atsushi