16:14:04 RRSAgent has joined #immersive-web 16:14:04 logging to https://www.w3.org/2020/11/04-immersive-web-irc 16:14:19 RafaelCintron has joined #immersive-web 16:14:40 zakim, add agendum 1 Spatial Capture (Ben Erwin?) (60 minutes) 16:14:40 I don't understand you, atsushi 16:14:50 agenda+ Spatial Capture (Ben Erwin?) (60 minutes) 16:15:27 agenda: https://github.com/immersive-web/administrivia/tree/main/TPAC-2020 16:16:10 zakim, agenda order 7, 2-6 16:16:10 ok, atsushi 16:16:14 zakim, list agenda 16:16:14 I see 6 items remaining on the agenda: 16:16:15 7. Spatial Capture (Ben Erwin?) (60 minutes) [from atsushi] 16:16:15 2. Acknowledgements (20 minutes) [from atsushi] 16:16:15 3. Break (10 minutes) [from atsushi] 16:16:15 4. Lighting Estimation (@toji) (30 minutes) [from atsushi] 16:16:15 5. Break (15 minutes) [from atsushi] 16:16:16 6. Layers + Dom Overlay Hotspots in Head Mounted & Dom Overlay(@cabanier, @klausw) [from atsushi] 16:19:48 klausw_ has joined #immersive-web 16:21:37 scribenick: ada 16:22:02 Agenda Item: How to get acknowledgements: 16:22:34 Alex + Brandon: Discussion on how to collect acknowledgements and what counts 16:22:51 brandon: Every contribution counts, scribing is ++ count 16:23:19 bajones_: google form worked before 16:23:52 ... but any form that allows for automated collection/contributor driven collection 16:25:18 klausw has joined #immersive-web 16:25:28 bajones_: a github bot would be perfect 16:25:32 ada: I can build it 16:25:38 alexturn has joined #immersive-web 16:25:42 present+ 16:26:07 ada: what details do we need? 16:26:53 bajones_:we've taken emails before but maybe any contact will work 16:27:17 bajones_:we at least want company 16:31:37 ada: @bajones could you add a html comment ican hook into and i can make a bot that will turn acknowledgement issues into PRs 16:31:57 bajones_:how should we distribute the call for acknowlegemenrts/ 16:32:09 i/Agenda Item: How to get acknowledgements:/topic: Acknowledgements (20 minutes)/ 16:32:15 rrsagent, make log public 16:32:18 ... last time was on the mainling list but maybe not wide enough, how about a comment in the bs? 16:32:21 rrsagent, publish minutes v2 16:32:21 I have made the request to generate https://www.w3.org/2020/11/04-immersive-web-minutes.html atsushi 16:32:36 ... it maybe too noisy if we do that. 16:32:46 ada: if we get too much spam we can just take it out. 16:33:04 ... we can probs move onto the next topic 16:34:43 zakim, take up agendum 3 16:34:43 agendum 3. "Break (10 minutes)" taken up [from atsushi] 16:36:30 taking a break since we had to skip one issue and now are running fast 17:02:23 ada: are we starting again at 9? 17:02:33 That's what I thought 17:04:23 Slides for lighting estimation topic: https://docs.google.com/presentation/d/1kvmA6YGWhE6bwoCa9OYNxIJ_skA_pYFAHrkKJPRQglw/edit?usp=sharing 17:04:33 (Also linked in the agenda) 17:05:16 bialpio has joined #immersive-web 17:06:30 zakim, take up agendum 4 17:06:30 agendum 4. "Lighting Estimation (@toji) (30 minutes)" taken up [from atsushi] 17:11:08 lgombos has joined #immersive-web 17:11:17 Present+ Laszlo_Gombos 17:22:32 q+ 17:26:10 ack RafaelCintron 17:26:13 scribenick: cabanier 17:26:39 RafaelCintron: what are the numbers that are returned? 17:26:46 ... and what colorspaces are they in? 17:27:15 bajones_: for arcode, every color is 16 bits 17:27:22 ... I still need to investigate more 17:27:37 ... the floats are bounded from 0 to n, not 1 17:27:47 ... it seems that there are assumptios 17:28:03 ... it seems that there is a mapping to opengl and vulcan 17:28:17 ... it seems it doesn't map cleanly to srgb space 17:28:32 ... the raw blob of numbers shouldn't be surfaced up to the web 17:28:38 ... it will come back as a texture 17:28:53 alcooper has joined #immersive-web 17:29:00 RafaelCintron: if it's just an array of number, what is the height/width 17:29:11 bajones_: it's a 16x16x16 cubemap 17:29:17 ... so it's not a problem internally 17:29:28 ... it's to contrast with arkit which hands you a texture 17:29:44 ... so either metal has to do the conversion or you have to extract them yourself 17:30:07 ... we'd rather have the texture so systems that use that aren't disadvantaged 17:30:21 RafaelCintron: what if you call it, what is the lifetime of the texture 17:30:32 bajones_: I think it needs to be a new one every time 17:30:47 ... especially if mipmapping needs to happen 17:31:05 ... so if the developers changes it, we don't want to leak thing 17:31:17 ... which means we have to return a new one 17:32:03 ... since it only happens once a second and in response to an event, it's not a big deal 17:32:14 ... authors shouldn't pull it in every frame 17:32:26 ... they should listen to the event and then do the right think 17:32:52 RafaelCintron: wrt the light probe, and a long time later you passed it in, are you getting information from the past? 17:33:04 bajones_: no, the light probe should update over time 17:33:14 ... arcore only has 1 light source 17:33:38 ... and it's attached to the viewer and basically gives the light conditions around your phone 17:33:56 ... in arkit, you can place a lot of spaces and they are tracked over time 17:34:28 ... you can say which probes should be updated so they are tracked when the user moves 17:34:40 ... which is something you can do on iOS 17:35:01 RafaelCintron: so when you call it repeatadly, you get the same information? 17:35:18 bajones_: yes. Until recently, you would even get the same object back 17:36:03 ... if you called it. Now that I added options it's no longer the case 17:36:04 q? 17:36:34 ... there is a repo for this and it has an open PR 17:36:56 ... Manish has given the thumbs up but I'd like to get the opinion of more people 17:37:58 ada: it would be great if we can get someone from webkit 17:38:10 ... has dino been involved? 17:38:31 bajones_: at the Seattle F2F, I asked Apple about that. 17:38:45 ... and after that, the documentation suddenly got better 17:39:39 ada: if there are no other questions, should we move onto the next topic? 18:15:53 DaveHill has joined #immersive-web 18:16:33 Slides: https://docs.google.com/presentation/d/1XJeAJs5DQ8teAeKL1ujJdjByn3ReOH_IRfu3wwXGsAY/view 18:20:00 alexturn_ has joined #immersive-web 18:20:15 scribe:Alex Turner 18:20:21 present+ 18:20:28 present+ 18:20:33 present+ 18:20:41 s/scribe:Alex Turner/scribe: Alex Tuener/ 18:20:48 scribenick: alexturn_ 18:21:18 (will drop shortly around 3:30am, go to sleep) 18:21:30 s/scribe: Alex Tuener/scribe: Alex Turner/ 18:23:27 I think full interactivity is a use-case we care about in the VR flavor of this. 18:26:15 q+ 18:26:20 q+ 18:27:59 q+ 18:29:20 ack alexturn_ 18:31:38 q+ 18:32:34 alexturn_: screenspace and headmounted dom overlays have different user requirements. 18:35:15 ack cabanier 18:36:30 cabanier: how are these annotations done, through a script or the UA? 18:36:52 klausw: a script 18:37:06 how this will work in stereo presents many issues 18:38:03 ack ada 18:38:53 q+ 18:39:03 q+ 18:39:13 ack bajones_ 18:39:17 q+ 18:43:52 ack alexturn_ 18:44:24 bajones_: we could provide an option to facelock the UI 18:44:45 alexturn_: what would stop developers opting in by default 18:45:12 q+ to ask about facelock plane distance 18:45:38 bajones_:we need to make it a little unpleasant so people know it should not be the defualt path 18:46:30 alexturn_:whats the security implication behind 2 dom layers? 18:47:11 klausw: issues regarding user surprise as they may not tell what is dom vs webgl 18:48:01 q+ 18:48:28 alexturn_: proposing an extensio nto support world positioned dom overlays 18:48:32 ack cabanier 18:48:41 scribe: alexturn_ 18:48:56 alexturn__ has joined #immersive-web 18:49:18 q+ to respond to klausw 18:50:54 ack klausw 18:51:36 alexturn has joined #immersive-web 18:51:44 scribenick: alexturn 18:52:10 q+ 18:52:34 klausw: Folks focusing on phones won't use an API if it's more complicated 18:52:43 klausw: For non-interactive quads, why not just handdraw it yourself? 18:52:45 q+ 18:53:15 q+ 18:53:27 q+ RE: why not just draw it yourself on a quad 18:54:10 klausw: If you had a head-attached quad, it would not be easy to interact with 18:54:39 klausw: 1. Do we have what we need to make non-world-attached DOM overlay work well 18:55:05 klausw: 2. For world-attached content, what do we need for apps that make screen-space assumptions? 18:55:22 ack ada 18:55:22 ada, you wanted to ask about facelock plane distance and to respond to klausw 18:56:16 ada: For detached elements, if you had an interface in your 2D browser where the buttons are slightly popped out in the Z direction, and that interface is carried with the user into the environment, the user would expect the buttons to retain their 3D aspect 18:56:36 q? 18:56:39 ack bajones_ 18:58:17 bajones_: If you do DOM overlays across multiple overlays, we could perhaps deal with the privacy impact, though there is still a concern 18:58:27 bajones_: We need a more dynamic API than the current "set it and forget it" fullscreen API for one overlay 18:58:35 bajones_: You'd need to move them around 18:58:52 q? 18:58:56 ack RafaelCintron 19:00:14 ack RafaelCintron alexturn 19:00:21 ack alexturn 19:00:51 alexturn: people have to do maths to place stuff where they want it 19:01:16 alexturn:for the dots people have to work it out themselves 19:01:41 ... if we can provide a hook for them to calculate it we would get people onto the right path 19:02:15 alexturn: for the headlocked version, we might need to think about the privacy issues 19:02:36 q+ 19:02:43 ... but I am not sure I can see what they are the risks seem the same to me 19:02:57 q? 19:03:00 ack cabanier 19:03:23 cabanier: If you render the DOM overlay on top, how do you render the controllers? 19:04:19 cabanier: For detached elements, there's something you could do there, but it would be hard to specify how to do that 19:05:35 ada: Rolled a 9 to pick a scribe 19:06:07 scribe: Lachlan Ford 19:06:30 Lachlan_Ford has joined #immersive-web 19:06:48 scribenick: Lachlan_Ford 19:06:51 q? 19:07:02 ack DaveHill 19:07:02 DaveHill, you wanted to why not just draw it yourself on a quad 19:07:31 does the group have a way of thinking about phone-only vs. all-device features 19:07:37 do we have principles around this? 19:07:41 q+ 19:07:48 I feel like the gist of Brandon's argument is "people are going to do this anyways" 19:08:01 ack alexturn 19:08:24 alexturn: As a headset vendor, principle is all device features. Failure if by virtue of using a feature you've built a phone only app 19:08:46 ... Except for transient input, which you wouldn't get on a headset there isn't too many things (inline excepted) 19:09:16 DaveHill: This is the first feature that feels hard phone only oriented 19:09:38 ... Feels like we're going down a path of adding phone only features, would we apply that to headset only features? 19:09:57 Ada: In that situation, developers want to use the feature in VR because you get a significantly better result 19:10:28 ... HTML is valuable for historical reasons 19:10:39 DaveHill: The DOM overlays feels very phone oriented 19:11:14 ... Not pushing on the use of 2D HTML, the issue is that it doesn't feel as though it translates well to headsets without 3D translations we're unsure of 19:11:29 AlexTurn: I'd push back on anything about "having UI" being phone oriented 19:12:02 ... Rolling your own UI is a pit of failure for devs, so need a path to have it work universally 19:12:48 ... Need to ensure we use our power for good and dissuade developers for writing head locked content that is not universal 19:13:03 q? 19:13:06 ack klausw 19:13:39 Klausw: The DOM overlay API is a compromise for how to do DOM in XR. Not perfect for all usecases. Solves some important usecases 19:14:09 q+ 19:14:16 ... Clear intention to allow simple features like a start button / model selector to work on the headset without the app dev needing to do anything specific to support headsets 19:14:44 ... "How can we make this not completely break on headsets and what should developers do instead" 19:14:59 ... Questionable whether a more complex API would be attractive to developers to use 19:15:13 ... Fundamental differences in UX between phone and HMD modalities 19:15:23 ... Hard to have an API which for solves both 19:15:31 s/solves both/solves for both 19:15:59 ... There are expectations, e.g. no input events for cross origin content 19:16:06 ... Will need UI hints to indicate that 19:16:26 Rik: If you hit the DOM overlay your controller disappears? 19:17:06 ack alexturn 19:17:08 Klausw: Spec language about the host content not having access to XR input data, so will need UI hints e.g. reticle 19:17:41 AlexTurn: DOM overlays are doing a good job as a honeypot to prevent devs from falling into bad patterns like screenspace 19:17:55 ... is there incentive to use further API vs just doing screenspace ? 19:18:23 ... Overylays you showed were independent in the world. We could do a lot of work for devs to simplify the API 19:19:07 ... For input routing, similar to system UI (controller / hand is captured by system and not routed to apps) 19:19:17 ... For DOM overlay we have the same problem 19:20:02 ... UA would have to route input away from Apps based on where it is pointing and what content it is aiming at 19:20:13 ... have to solve this problem anyway to be ready for multiple overlays 19:24:49 Boo! I want the DOM to stretch around me as an equirect! 19:24:52 ;) 19:29:08 q+ 19:29:41 ack RafaelCintron 19:32:19 q+ 19:32:23 q+ 19:32:24 q+ 19:32:25 q+ 19:32:26 q+ 19:32:31 ack ada 19:34:07 q+ 19:34:24 Ada: Suggests layer would load about:blank and then content would be added 19:34:38 AlexTurner: Would want to load this way with transparent content initially 19:35:00 ... Start from some kind of new tab page until switchover can be seamless 19:35:11 ... Sibling property to set the background color might be meaningful 19:35:28 Rik: There was a metatag proposal to set whether the window is transparent 19:35:57 AlexTurner: If transparent and took a few frames to load things it'd be fine. If it's white then takes a few frames it would be noticable 19:36:11 Rik: Transparent windows are allowed so can spec it without breaking any assumptions 19:36:14 ack klausw 19:37:07 Klausw: Like it, more flexibility but restrictions on the DOM content to make it work. Not allowing 3rd party iframes seems like a good tradeoff. How do nonflat HTML elements work? (e.g. select) 19:37:19 ... Might there need to be restrictions? 19:37:28 Rik: Don't they all get rendered to the same surface? 19:37:39 Klausw: No. e.g. on Android its a different UI. 19:37:50 Rik: Delegate render to a different surface? 19:38:01 Klausw: On Chrome it is platform specific 19:38:32 AlexTurn: Edge may run in different modes. Separate mode which constrained itself to the window border for e.g. windows phone. This mode was used for HoloLens 19:38:39 ... May have solved the relevant issues there 19:38:58 ... Whatever solution for e.g. comboboxes may be transferable to this domain 19:39:29 Klausw: If you want to impl DOM overlay, you need a full feature browser engine because there is no restriction on DOM 19:39:37 ... With restrictions, would not need the full engine 19:40:00 ... Another issue is system UI, e.g. does the DOM cause virtual keyboards to appear? 19:40:29 Rik: If you pop up a window, your controller poses go away and you use the virtual keyboard with system controllers. 19:40:44 ... Users would see the window popping up and see different controllers appear 19:40:47 ack bajones_ 19:41:17 bajones_: What determines pixel size (css?) 19:41:33 ... What is the semantics of width and height. What resolution does the page render at? 19:41:59 Rik: App needs to determine meters to CSS pixel mapping 19:42:10 bajones_: Not sure there is a significant downside to mapping to meteres directly 19:42:27 ... matching the aspect ratio of the in-world element seems reasonable 19:42:34 q+ to mention CSS centimeters and meters 19:42:46 ... Don't want to leave it to UA's as the incentive as resolutions get higher, they'll want to scale everything up 19:43:38 ... Proposal as is prevents you from using DOM from the 2D page itself. 19:43:59 ... How do you determine whether I'm interacting with DOM vs World? 19:44:16 Rik: Hit-test from pointer to DOM content 19:44:43 Klausw: Could capture onselect with DOM elements if they contain listeners for it 19:45:16 ... Could allow event cancelling 19:45:36 AlexTurn: If you hover away we end up taking the pose away, to prevent double rendering from multiple layers 19:45:53 ... Layers not always on top 19:46:07 ... which means app content may obscure it. 19:46:19 ... Interesting negotiation between app and system around it 19:46:51 bajones_: We could ensure that any time you're interacting with the DOM the system MUST render a cursor 19:47:19 ... Expect app to render its own controller in its own way, but don't allow the app pretending to point their ray somewhere its not actually pointing to trick the user into clicking DOM 19:47:35 q+ to ask about equirect domlayer 19:47:38 ... Should have the option to NOT capture the input (e.g. tooltip) 19:47:43 ... waste of resources and unwanted 19:47:45 ack RafaelCintron 19:47:46 q+ 19:48:17 Raphael: WebGL wanted an extension to draw DOM to WebGL textures 19:48:32 ... Similar problems. CORS limitations 19:48:49 ... Youtube would not work in that case (or gmail or office) 19:49:05 ... +1 to manditory cursor suggestion 19:49:08 ack alexturn 19:49:56 AlexTurn: Paused based on initial scenarios being CORS related (e.g. loading youtube) which halted discussions previously 19:50:16 q- 19:50:17 ... Good place to start with just constructing a UI using DOM 19:50:47 ... If this API existed and DOM overlays didn't, developers on phones would just make 1 quad and fill the screenspace rect 19:50:58 ... No they've unwound headset compat 19:51:04 s/No/Now 19:51:28 ... Here you could do both explicity, but we've lost the incentive for phone devs to fall into the right pattern 19:51:51 ... orders of magnitude more phones to expect content to come from there which jeopardizes future compat with hmds 19:52:28 ... Is there a role for a medium / high policy version closer to DOM overlays in addition to this? 19:52:32 q+ 19:52:52 ... Game engines want means to render 2D UI 19:52:55 ack Manishearth 19:53:15 Manishearth: Have concerns on architecture 19:53:30 ... If creating new window objects not from iframes, browser developers won't like it 19:53:43 ... iframes are already weird in terms of implementation and spec 19:54:04 ... window objects that have rendering behavior implies a lot of things that need to be explicitly spec'd 19:54:37 ... Wondering if there is another way to go about this 19:54:44 q+ about scalability - hundreds of annotations? 19:54:51 q+ to say about scalability - hundreds of annotations? 19:55:31 ... not allowing cross origin simplifies things but still have to worry about rendering capabilities of the device 19:56:09 ... moving windows around requires complex rendering behavior, which is cause for alarm. Need to discuss this. 19:57:32 ... Should consult low-level iframe / window infrastructure people and also the TAG 19:58:03 ... Is potentially tangled with other spec's, not an XR specific thing 19:58:07 ack ada 19:58:07 ada, you wanted to mention CSS centimeters and meters and to ask about equirect domlayer 19:58:38 Ada: CSS already has a concept of units 19:58:50 ... e.g. meters 19:59:43 CSS also says 1in == 96px. Which... 😒 20:00:32 ... Other topic is equirect layers. Not sure what would happen there 20:00:55 bajones_: cylindrical layer, wrap around 360 deg. Context in a tube around you, no need to deal with poles 20:01:13 ... Don't want to encourage combing coconuts 20:01:20 q/ 20:01:22 q? 20:01:25 ack cabanier 20:01:45 Rik: a css cm is not always a cm, its in pixels and depends on px / cm 20:01:59 bajones_: css px are not always device px 20:02:29 Rik: cross-origin usecase could be made to never receive input, and just be a dead texture 20:02:34 q+ 20:02:44 ... in that case you'd not get access to window contents, only the layer metadata 20:03:17 ack klausw 20:03:17 klausw, you wanted to say about scalability - hundreds of annotations? 20:03:31 I was concerned about this too, Kalus 20:03:40 klausw: People are going to use scalability in odd ways, which may make browsers unhappy. Need a gl style min / max limit for implementations 20:03:48 ... e.g. min = 4 20:04:13 Rik: Nothing preventing layers being huge. No min / max. Only limit is supporting 1 projection layer 20:04:35 ... One of the issues with windows, new process / memory 20:05:01 q? 20:05:06 ack bajones_ 20:05:10 zakim, close the queue 20:05:10 ok, ada, the speaker queue is closed 20:05:23 +1 to having some kind of minimum limit 20:05:24 bajones_: There exist reasonable limits for the base layers API 20:05:42 ... Shouldn't allow terrapixel large quad layers 20:06:06 ... pixel content being determined by pixel size in meters may not be a good idea 20:06:23 q+ 20:06:33 ... Better to push our content and make it huge then put it close to your faces 20:06:58 We just need an angular resolution CSS measurement 20:07:03 ... Implies IMAX res screens. Nobody wants to render their web content that large 20:07:12 pixel density feels like the right way to handle this -- UA can determine CSS to physical pixel scaling factor 20:07:21 ... thus caps on size + control from the user is warranted 20:07:55 Rik: Assumed transform would cover that but scale requires more 20:08:30 ... Would want to scale the normal way (maintain aspect ratio in all cases) 20:08:42 bajones_: Maintain aspect ratio and reflow content, even if more expensive 20:09:32 zakim, open the queue 20:09:32 ok, ada, the speaker queue is open 23:35:12 rrsagent, publish minutes v2 23:35:12 I have made the request to generate https://www.w3.org/2020/11/04-immersive-web-minutes.html atsushi 23:59:59 meeting: Immersive-Web WG/CG TPAC Day 3 (Wed) 23:59:59 previous meeting: https://www.w3.org/2020/11/02-immersive-web-minutes.html 23:59:59 i/alexturn_: screenspace and /scribe: ada/ 23:59:59 i|https://docs.google.com/presentation/d/1XJeAJs5DQ8teAeKL1uj|topic: Layers + Dom Overlay Hotspots in Head Mounted & Dom Overlay 23:59:59 s/Alex + Brandon:/Alex+Brandon:/ 23:59:59 s/how this will work in stereo presents/... how this will work in stereo presents/ 23:59:59 i/alexturn: people have to do maths/scribenick: ada/ 23:59:59 i/cabanier: If you render the DOM overlay/scribenick: alexturn/