Immersive-Web WG/CG (extended) group call - Oct/2021, Day 2

15 October 2021


ada, alcooper, alexturn, atsushi, bajones, bialpio, cabanier, cwilso, dino, Leonard, yonet
ada, alexturn_, bajones, bialpio, cabanier, cwilso, dino, laford

Meeting minutes

<bajones> (Come at me zakim!)

<cabanier> did I call into the right meeting?

<cabanier> thanks. I think I dialed yesterday's first

<bialpio> heads-up, looks like the calendar at w3 already lists our today's mtg as "past event" so if you don't see it, go to "View past events"

<atsushi> ah, that's an existing issue, that we cannot view an event which is running. not in upcoming nor in past...

<atsushi> (has fixed??)

<ada> https://github.com/immersive-web/proposals/issues/71

New proposal ideas?

<ada> https://github.com/immersive-web/proposals/issues/70

ada: this is the new proposals idea issue - anything we might want to add to the new charter, or keep in the background.
… something I already added is supporting 3d screen based hardware; do we want to add that to our charter.
… does anyone want to suggest new things?

piotr: should we think about working on point clouds? native apis have some capabilities about this; seems like it falls under RWG already, so probably don't have to add to charter explicitly

nick: I would be generally supportive of point clouds; we do have a whole session to discuss later

ada: how do people feel about adding 3D screen support, like dSpace etc

bajones: if something isn't in the charter, can we talk about it?

ada: not off the table; ...

nick: for 3d displays, seems like the views api would support this?

chris: if something isn't in the charter - not mentioned int he scope - it's probably not covered by the patent policy. If you want the group to own it, you should probably make sure it's covered in the charter.

alexturn: mentioning zspace thing - I would be supportive of adding that.

leonard: can the charter be less specific?

cwilso: charter can be vague - you're essentially licking the cookies, whether you are eating them or not (by putting them in deliverables)
… Members may object to the charter because of that expanded scope, though.

brandon: it feels awkward to say we're going to put in the charter an entire new category of devices in order to potentially add one new flag.
… it's probably rolled into 3d css or the like,.

ada: so maybe we should not have a specific deliverable, but say we're going to support immersive devices like immersive display technologies.


<Zakim> atsushi, you wanted to discuss (just comment...) we list some potential specs in current charter as under incubarion in IWCG, and I feel ok if we could include potential ones like them (at least) with one line of target area(s)...

atsushi: in current charter we have several lines of incubation, so I feel it's better to include emerging areas through incubation.

ada: that's a good idea.
… we do have a lot more flexibility through the community group.
… CG isn't covered by our charter.

nick: mostly +1 to brandon, such a small patch on the existing spec (maybe not even a patch, just in implementation). Understand process in general, but maybe let others know we have this covered. It's basically mostly done - you might initially implement it as a desktop session.

cwilso: should put in the charter; it's not a big deal, but we want to ensure it's in scope for patent policy reasons.

ada: this hardware didn't exist last charter, so we should lick this cookie.

alcooper: expanding scope is good; I think we shouldn't say we don't need to update the spec at all, devs probably need to have more information. Non-normative notes, at least.
… rendering to these different devices might be a little bit different

leonard: two items: audio; input from users' environment (note privacy concerns), and audio output (may be covered?). and 2, haptics.

laford: on haptics, openxr does have support for haptics, but it's not that exciting - buzz-at-freq-for-time

ada: do we want to add haptics?

cwilso: isn't this covered by gamepad?

alcooper: yes, seems sufficient for our scenarios

ada: what audio would we need?

leonard: don't know

laford: haptics is a volatile rabbithole.

<Zakim> atsushi, you wanted to discuss (just comment, no voice needed) non-normative notes are free from IP and we usualy state just as "Other non-normative documents may be created such as:" in charter

atsushi: non-normative notes are free from IP commitment, so other documents could be created (that aren't Recommendations, and don't imply Patent Policy)

<Zakim> ada, you wanted to mention haptic wear hardware

cabanier: we did request enhancing haptics on gamepad, but didn't get much traction

ada: one thing I've heard g4merZ talk about is haptic devices - like haptic suits or gloves.

brandon: I don't have visibility in every market, but I don't see too many efforts in haptic suits or gloves that are credible yet.
… what is relevant is more full-body-tracking. You can kind of stitch together today with the Vive tracking pucks.
… I don't know what we would need to represent that
… Also facial recognition, lot of research in pulling skeletons out of this data...

<cabanier> +1

brandon: in a totally non-creepy and non-threatening way...
… seems more realistic than haptics
… might want to add full-body-tracking to the charter

ada: I note we have loads more discussion before we send this charter in for approval.

laford: full-body tracking sounds like part of a subset of human understanding, which seems good to add.

OK, and on to issue 71

Potential AR feature - point clouds

bialpio: Last gaps between web and native APIs we have

bialpio: Is there much interest? Or is it too low level? No concrete proposals atm but we can work towards exposing info about point clouds underlying XR systems use to derive higher level features

bialpio: What does the group think?

Leonard: point clouds are potential GB in size

Leonard: how do you deal with that much data in memory or on the network?

Leonard: Is the tech fully ready yet?

Leonard: Geospatial people are working on static point clouds. Apple consumer devices have lidar point cloud

Leonard: consumer may be in the order of hundreds of MB

Leonard: ISO is working on point cloud streaming

Leonard: Idea is good, its a dangerous area to unwittingly wander into

<yonet> yonet +

Nick-8thWall: For us having a basic data structure would be useful

Nick-8thWall: our curved image target tracking within an immersive session requires extracting feature points so having access to a precomputed set of feature points directly would make our implementation much easier / cheaper

Nick-8thWall: we could build curved tracking on point cloud and image texture api much more cheaply than raw

Leonard: Is this data point cloud lidar or something separate?

Nick-8thWall: The point cloud collected by the device for tracking, whether lidar or image feature detection etc...

Leonard: Would be useful to structure it in such a manner to not deal with data sets but device collected data

bialpio: should look at it through the context of xr session. Would be a matter of exposing the data that xr systems are tracking currently to synthesize the higher level features

bialpio: would preclude working on big datasets

<Leonard> +1 for device tracking at the moment

bialpio: could be a point of simplification for the API

bialpio: only looked at it through the lens of AR

bialpio: is there an OpenXR equivalent

bialpio: how would this be implemented on as many devices as possible?

laford: no mechanism in OpenXR, for WMR devices it may be part of the secret sauce

alexturn: Not necessarily the secret sauce but may just have been something we did not want to give out

alexturn: dont want devs taking too hard a dependency on the specifics on the point cloud

alexturn: e.g. training an ML model on hardware A and it doesn't work on hardware B

alexturn: Perhaps for ARCore/Kit it has stabalized enough to not worry about this?

alexturn: for hit-test apps that would have been agnostic may lock themselves in by choosing point cloud

alexturn: do user agents have to pretend they have the wrong type of data for compat?

alexturn: may be approaching the point of technology specific but could accelerate algorithm development e.g. Nick's algo

Allow dynamic frame timing

<ada> https://github.com/immersive-web/webxr/issues/1233

cabanier: Want to talk about optimizing how we deal with frame rendering

cabanier: in Oculus we get all poses and ask for a new texture. If not free, have to wait.

cabanier: OpenXR has a way to solve this

cabanier: with webxr layers you could kind of mimic it but its not explicit

cabanier: Oculus also has "phase sync". Which adjusts the display time based on your rendering overhead to minimize latency

cabanier: need to know the rendering stages though

cabanier: could make it explicit for layers

cabanier: Would like to experiment with it

cabanier: is it already working on android?

laford: Cass Everitt big driver of this on the OpenXR side

RafaelCintron: No objections in principle. Will need to restructure things to get pose data separate from rendering data

RafaelCintron: In a browser architecture, how much difference will it make?

RafaelCintron: If it is affective, seems like a win

bajones: Right now there is separate mechanisms in the layers API for headpose and texture. Would that be sufficient?

bajones: Feels like something that ideally should be best practice

cabanier: Could be a normative note?

bajones: having a note "if you spread out these calls backends may be able to rendering" seems like the basic step we should do here

* optimize rendering

Nick-8thWall: One thing to be aware of; We depend on the order of threejs operations. If these things get change in a material way, we'd want to make sure our experiences aren't broken

cabanier: Should be optional as to not break existing experiences

Nick-8thWall: The threejs implementation is a little too turn-key with no ability to manually drive it. Have to hook into the threejs systems that are restrictive

Nick-8thWall: If we are rethinking threejs / webxr APIs maybe we should give more control to external apps and leave rendering to threejs

Nick-8thWall: example: "responsive scale", where experience are layed out in front of you where the dev intends them to be. We rely on threejs to update its own structures based on webxr poses, and we have to MITM them before threejs uses them

Nick-8thWall: Were they separate and control fell to WebXR this would be more convenient

alexturn: Worth pushing to see how far you can get. OpenXR is general based on how much runtimes want to optimize. Tricky to update pose half way through the frame causing a synchronous round trip all the way up and down the stack

alexturn: potentially an API for apps to ask for update poses. Things may be different in WebXR due to Asynchronicity

<Zakim> bajones, you wanted to speak to three.js integration a bit

bajones: Responding to Nick: Threejs does not directly control the direction of threejs. I agree that threejs is a little too turn-key.

bajones: the threejs devs like this and like the API to be opinionated. Coordination based on large scale restructuring may be tricky but they could be pursuaded

<RafaelCintron> Ada, have you heard of Babylon.JS?

<ada> RafaelCintron:of course :)

<ada> I also like PlayCanvas

<ada> There is also wonderland

cabanier: Replying to Alex: Used to offer getting updated poses during lifetime of frame, Didn't that go away in OpenXR?

alexturn: When you call xrLocateSpace you get the latest answer based on the time, so for the same time may get a more update pose later

alexturn: can leverage this for rendering by calling it at a later time

alexturn: we rely on locate space being instant, whereas in WebXR there is a whole process model that changes the dynamic

alexturn: xrFrame is more of the explicit model we used to have. Could see an xrFrame API for "getUpdatedPositions" which gives you a new xrFrame

Feasibility and desirability of persisting anchors

<ada> https://github.com/immersive-web/anchors/issues/69

ada: anchors are amazing and I want to share them
… or hold onto them and share them later
… some platforms have shareable anchors
… they could be stored in the browser and used later

<Leonard> 1+

alexturn: there are transient anchors for making things more stable
… there are persistent anchors for use across sessions
… and there are shared anchors which have a cloud backend which are hard to standardize
… unless UAs support all backends, it would be hard to standardize
… magic leap lets you find the nearest anchor and you find the nearest post to that anchor
… MS lets you create an anchor and use it

Nick-8thWall: I was going to say the same thing that alexturn said
… the cloud based spatial location is more application level and not so much within the browser's scope
… there are use cases for multi session but it seems less of a use case
… I don't think it's the highest priority

bajones: I was going to ask alexturn and Nick-8thWall to ask what OpenXR is doing
… we want to get them on the same page
… fundamentally it will be hard to standardize on our level
… because it is very platform dependent

<Zakim> ada, you wanted to ask about just leaving the anchor list populated

Leonard: could the developer turn off anchors

ada: are persistent anchors a stepping stone to shareable ones?
… if you have anchors enabled and you go to the same domain, you can find them again

alexturn: we could find something that could be persistent across sessions
… I don't know if it's trickier for phone based AR
… sharing is much more difficult
… for instance we have different APIs for persistent and cloud API
… persistent anchor are a native API

Nick-8thWall: I could walk through to the computer vision
… if there's a key attached to an opaque blob
… what does the blob contain?
… there's a bunch of tech that goes into that re-identification
… there are image level hashes to re-identify scenes
… a lot of these technologies have magic
… for instance google and MS use very similar technology
… so their token might not be understood by each other
… (point clouds)
… even being future compatible with yourself is hard
… so sharing across devices and vendors is hard
… even across session that are hard problems to be solved

bialpio: talking about the 3 tiers
… on arcore you always need to go to the cloud to persist something
… it seems that there is something that's browser level
… but the big unknown is to find a way to works everywhere

alexturn: local persistence skips the vendor problem
… and that over time things change
… we don't hand the blob and just store things internally
… in that world, there's more a chance that we have a common API

<Zakim> ada, you wanted to suggest that maybe this is not the path

alexturn: right now we don't have a great way to share across

ada: it seems that persistence isn't the route towards sharing
… and not that useful of a feature at all
… last time we talked about this, we said we didn't want to share
… and this hasn't changed
… maybe for the next version of anchors we could define a meta format
… is this something that we want to do?
… or should we wait

alexturn: at the moment all the formats are opaque
… my suspicion is that this is churning
… we can't promise compat
… until things stabilize
… there was a body that was trying to align this across vendors
… but I don't think we're at that point yet

bialpio: we might want to think about the use cases
… I'm pretty excited about marker tracking
… so you can localize multiple people at the same time
… it can certainly help with thatr

<yonet_> https://www.openarcloud.org/

alexturn: openarcloud.org was the group but it doesn't seem to have updated since 2019

HTMLModelElement https://github.com/immersive-web/proposals/issues/69

<dino> i will be there in 5 minutes

<dino> sorry that i'm late

dino: Recently, we made an explainer proposal for a <model> tag

dino: The idea is to promote 3D models to be at the level of image and video on the web

dino: Obvious issues are which file format to support, how to get consistent rendering

dino: We still think this is a really good idea

dino: One thing we think this is useful for is as a WebXR layer - might be useful as an easy way to put a 3D model into WebXR as a layer

Leonard: In general, in concept this is a good idea

Leonard: Helps folks incorporate 3D into regular web pages

Leonard: Still a tremendous number of problems trying to display a simple model

Leonard: Differences in look across browsers could be an issue

Leonard: Wish to see a WebComponent similar to the model-viewer component Google has

Leonard: I think this is good to implement as a WebComponent but not as a web element

<Zakim> ada, you wanted to ask about the charter

ada: To Leonard's point, I actually think the lack of consistency across browsers is a feature, not a bug

ada: Browsers can adapt the content to the limitations of the system

ada: A developer in WebGL may not pay attention to the specs on a low-power device and could miss devices that can otherwise handle that content

ada: Dean, do you think this would be a good thing to go into the charter?

dino: I would love to see this go into the charter.

ada: Cool!

bajones: Agree that this belongs in the charter.

<dino> I'll wait until everyone speaks and then try to answer :)

bajones: I've been somewhat vocal in my disagreement with this proposal online - I just want to be clear about the goals we're achieving here.

bajones: Not sure if a full tag is what we should do first here.

bajones: Agreed that WebGL is hard! I know it well and also find it difficult building compelling content.

bajones: I do wonder whether the folks who would use this <model> tag are comparing it to the model-viewer component

bajones: Should just be one include as the difference

bajones: Whereas model-viewer has a lot of extra capabilities vs. what this proposal gets you

bajones: If we move forward here, let's focus on the delta in what you can get compared to model-viewer today

bajones: Ease of use is not likely one of those

bajones: Better performance, faster parsing, better battery life could be key advantages here

bajones: Not sure if we have these numbers yet.

bajones: Knowing the key advantages is important here

cabanier: One key advantage is that you could get the models to actually show up in 3D

<yonet_> https://modelviewer.dev/

cabanier: WebXR needs permission prompts to get head poses - <model> would solve those issues.

cabanier: Cross-origin stuff could be solved here

cabanier: Scrolling can be difficult to handle smoothly - native handling could solve that

cabanier: How big can it be? Is it constrained to a little area? OS could manage constraints there

cabanier: This would start simple but could solve issues over time

alexturn__: I agree with Brandon

alexturn__: to the points which cabanier was making, one key difference is the user viewing it on a headset or a falt display
… what benefit are we getting from the component, for flat displays the benefits are ux like battery and performance. But for immersive hardware it's a huge ux benefit because it can actually have depth
… what if people want to drag things out like you can with images in web browsers today. You could then do that with the model without the page needing to understand it
… can drag it to other applications or to your desktop
… there are many benefits we can add in the flow of the page for a 3d model

<yonet_> ack ..., lighting, etc.

<yonet_> ack ...

dino: Alex and Rick said many of the things I was thinking here.

dino: Definitely useful for headsets - even for phones, folks have asked us for 3D models within 2D WebViews within AR scenes.

dino: Consistent rendering is a key thing to solve - MaterialX could help here as it tries to tackle that

dino: Battery life and dragging out of the page are interesting too

Nick-8thWall: Scenario of embedding 3D content within floating 2D WebView is indeed interesting

Nick-8thWall: Coming at this from a different perspective on the developer side

Nick-8thWall: I generally want to give power to developers here - if developer can do it or browser can do it, I want to give power to developers

Nick-8thWall: Developers rarely want to stop at simple 3D content - they want interaction as well

Customization is important too, even for a simple viewer

Nick-8thWall: Customization is important too, even for a simple viewer

Nick-8thWall: If we prematurely put things behind a <model> interface, we may cut off experimentation in that regard

Nick-8thWall: Is there another session type here as a looking-glass immersive session

Nick-8thWall: Could that be done programmatically?

Nick-8thWall: In terms of efficiency, I think there's a tendency to lean on these concerns too much

Nick-8thWall: We've seen great strides in SIMD in WebAssembly over time

Nick-8thWall: The web tends to get more efficient over time, whereas the perf concerns at the beginning of a standards process fade over time

Nick-8thWall: I would always advocate for tech that lets better products get built rather than a cookie-cutter mold

RafaelCintron: Dean shared a lot of my points

RafaelCintron: Permissions are simpler here - all you have to do is let things pop out, rather than allowing scene understanding, etc.

RafaelCintron: Accessibility can be better too - can have special modes that browsers support such as wireframe mode that don't require pages to participate

RafaelCintron: None of this should replace anything that exists

Leonard: Several people have mentioned lighting - we've looked at this extensively in Khronos

Leonard: In Commerce, people want the product to look exactly right - not just lighting but tone-mapping after rendering

Leonard: Without a reference here, I'd be very concerned

Leonard: Folks at Google have talked about getting the tone mapping exactly right with model-viewer

Leonard: People have talked about VR vs. AR - AR lighting is trickier

Leonard: Point lighting vs. other lighting can be an issue with PBR

Leonard: Format evolution can be another concern if some browsers don't support newer model formats

Leonard: If we can't have a path forward for model formats, feels way premature

bajones: Glad Leonard brought up the lighting work in Khronos

bajones: A lot of work to take PBR from glTF and add extensions for lighting that retailers find important and ensure it appears consistent in multiple renderers

bajones: We have tests to ensure that three.js and Google's Filament and Babylon are all within tolerance

bajones: This is important work that is already being done and it'd be a mistake to not lean on that

bajones: The model format that this thing hinges off of is going to be enormously important

bajones: If we end up in a situation where browsers can pick and choose which formats to choose and not have at least one shared format, that will be strictly worse than where we are today

bajones: It would be a very sad state of affairs if one browser supports one format and one supports another and not have one unified format

bajones: We don't want to repeat the mistakes of video tags past

bajones: Even if we have the <model> tag, we should make sure we can still experiment in that same space with lower-level primitives

bajones: We did talk about having inline sessions with some level of head-tracking

bajones: Has some issues and limitations around permissions

bajones: Also would create problems rendering outside the bounds of the page

bajones: Would be good to make sure we can still experiment outside the bounds of the tag

laford: Just wanted to voice support for the concept in general - really like the idea

laford: Rather than having WebGL separate from the DOM - having 3D models be part of the 2D DOM

laford: Even just having the ability to saying this is inline 3D content and seeing what browsers do with that

laford: Phones will treat it differently than desktop and different than headsets

laford: Moving content around can be enabled if everyone knows what is going on with the tag

dino: Just to be clear, this is definitely not trying to replace model-viewer or WebGL or anything like that

dino: I did research the model-viewer's excellent research into lighting in the explainer

<yonet_> yonet_+

dino: The lighting problems mentioned apply to WebGL viewers as well - we need to explore this no matter what

dino: Understand the need for a single format - video has one today even if not in the spec

dino: Not mentioning glTF just an oversight

dino: Even if you look at these different scene formats, they are fairly similar and I could imagine an API level here

dino: The explainer proposal would be just a start

bialpio: The permissions benefit only works if we don't let the app read back things that could be dangerous

<dino> good points bialpio! Internally we joke about this as "VRML5"

bialpio: Might start simple but could grow into something very complicated to support

<Zakim> cwilso, you wanted to address the two elephants in the room.

cwilso: Happy to see that one of the elephants in the room was brought up around formats

cwilso: Building standards is for improving interoperability - need a baseline of support

cwilso: If we come up with a <model> tag that can't be used consistently everywhere, we haven't made things more consistent

cwilso: Also worth noting that if we propose a new tag, this will ultimately just be a proposal to the WHATWG HTML WG

alcooper: Kind of wanted to go back a little bit to WASM and SIMD

<dino> +1 to what cwilso said. Can incubate here, but would have to go to WHATWG eventually.

alcooper: Looked into face mesh support in the WebXR APIs and a lot of developers told me the perf gains wasn't worth the tradeoff

alcooper: A lot of people brought up other things we just can't do that might be better tradeoffs

cabanier: Just want to note around performance that we are still adding things to the spec to fix performance - that will still be the primary gotcha for years to come

cabanier: For inline models, each session would need its own full-screen buffer - may be a non-starter

cabanier: Video tag has had format gotchas, but it wasn't fatal - things go on even with less common formats also supported

cabanier: Just to have 3D content pop out, I don't think we need permission prompts there

Nick-8thWall: Just want to respond to a few things

<bajones> re: Rik's comment about performance and needing fullscreen contexts - that would only be the case if you were rendering above the page. You could still do a lot of interesting work rendering behind it though a cutout window that's bounded to the canvas.

<bialpio> re Rik: correct, my point was that you need to stay declarative if you want to skip permission prompts

Nick-8thWall: To the point about performance not being there, just want to thank you for doing the work to improve perf for everyone - it does pay off for the community!

Nick-8thWall: Heard some conversations about declarative vs. imperative

<bajones> Also, certain limitations of WebGL won't carry over the WebGPU where, for example, you'll be able to have a single device driving multiple canvases.

Nick-8thWall: Heard the permissions issues around imperative APIs

Nick-8thWall: Ease of use is not really an issue since you can wrap declarative APIs around imperative APIs

Nick-8thWall: Just want to make sure we're honest there about the benefits

Nick-8thWall: Nobody believes we're here to remove WebGL/WebXR

Nick-8thWall: If there are things the browser can do in a privileged way, that can hold back innovation and land us on suboptimal solutions

Nick-8thWall: In the same way the web evolved from <image> to <canvas>, that should be the way we look here

Nick-8thWall: If we have some way to make things interactive, we can allow more innovation

Nick-8thWall: Need to be careful not to privilege innovation in browser over innovation in developer

alexturn__: Great discussion. I think we'd benefit from a clustering of topics

alexturn__: there is a spectrum of how convincing they are. i agree ease of use is the least convincing.

alexturn__: and yes, perf might be transient. the most important ones to me are about user experience. with bucketing of topics we might be able to easily identify these things. the explainer should do this.

alexturn__: e.g. img v canvas - even though canvas is a superset, it hasn't replaced img.

alexturn__: i suspect there is room for both approaches

ada: one big advantage we get from this is better accessibility. a gltf that has been marked up in a way that can be exposed to a screen reader would be a significant benefit.

ada: also, the ability to add annotations and click events into a <model> (like image maps) makes a lot of sense

laford (via chat): image -> canvas === model -> webgpu / webxr

Image: User interpretation of content, with all the browser bits and pieces (events, accessibility etc...)

Canvas: Developer wants this to behave and look a specific way so sacrifices the benefits of the tag

Analogously a basic model tag is the image and WebGPU / WebXR is the Canvas

ada: e.g. click on a entity that has some identification, that can be exposed to the accessibility system

<bialpio> (from meet chat): <video> is to WebCodecs as <model> is to WebXR ?

<bajones> +1

<cabanier> +1

<laford> +1

<RafaelCintron> +1


<bialpio> +1

<alexturn__> +1

<Leonard> What Ada just said implies (to me) that one or more example Web Components really should be developed first

<ada> +1

<Leonard> +1 for charter inclusion

<alexturn__> WebCodecs is sort of halfway between - I liked Dino's idea of perhaps having an API breakdown of a scene loader format which could then allow "model codecs" still with the benefits of the <model> tag

<alcooper> +1

bajones: <model-viewer> does support ids, hot-spots, etc. It loses some of that information when handing that off to the AR view on iOS.

yonet_: what formats does <m-v> support>? what would happen if i uploaded a CAD file?

bajones: mostly glTF. it's based on three.js so maybe you could show other models.

bajones: given a huge file, it will try...

<Zakim> bajones, you wanted to mention that model-viewer does have accessibility support and hotspots

laford (from chat): XR Scene Markup may find its time soon if it ends up being a markup mapping of stable formats like GLTF and USDZ

Leonard: the game is https://spookyball.com

Source is at https://github.com/toji/spookyball

<Nick-8thWall> "Your browser doesn't appear to support WebGPU. (Scary!) This game requires WebGPU support." =(

And the model I mentioned specifically is https://github.com/toji/spookyball/blob/main/media/models/graveyard-compressed.glb

alexturn__: <model> tag can become another backend to <model-viewer>

ada: Do any of the model loaders support progressive loading?

ada: Like progressive JPEG?

Leonard: Models have a front and back - a smart loader could decide not to load the back faces

Leonard: Turns out the biggest thing to avoid loading is the textures more than vertices

<bajones> https://doc.babylonjs.com/divingDeeper/importers/progressiveglTFLoad

ada: Could we load the small texture sizes and small vertex detail first?

Leonard: Can certainly do that work - generalized 3D case is tough to solve

bajones: Posted a link to Babylon's progressive glTF loading based on the MSFT_lod extension

bajones: Can also go grab specific mip levels manually in an imperative approach

Nick-8thWall: Around different model formats, one thing we find popular is different hologram-type formats

Nick-8thWall: At least 4 separate formats we support now

Nick-8thWall: Capture and streaming formats

Nick-8thWall: If they're not part of the standard, they'd lose out

Nick-8thWall: Having an ability to have different players would be very useful

bajones: Can you explain what you mean by "holographic"?

Nick-8thWall: Volumetric video formats

Nick-8thWall: Capture of a person played back in 3D

Nick-8thWall: Microsoft HCAP format based loosely on glTF

<Leonard> MPEG (within ISO) is currently working on standardizing things like that.

Provide statistics to help guide performance https://github.com/immersive-web/webxr/issues/1203

<Leonard> Thank you for all of the discussion -- very enlightening

Review WebGPU API design at TPAC https://github.com/immersive-web/WebXR-WebGPU-Binding/issues/3

ada: Dean, any additional topics for the charter?

ada: Ahh, schucks - he had to drop

<Nick-8thWall> Here are some examples of non-gltf:

<Nick-8thWall> https://www.8thwall.com/playground/hcap-relight

<Nick-8thWall> https://www.8thwall.com/playground/ply-loader

<Nick-8thWall> https://www.8thwall.com/playground/tetavi-threejs

<Nick-8thWall> https://www.8thwall.com/playground/depthkit-aframe

<Nick-8thWall> https://www.8thwall.com/playground/threejs-8i

<bajones> Slides! https://docs.google.com/presentation/d/1wXzcwB-q9y4T5VL9sKRhAWz_2nvwAWdFtTvVgk43rlk/edit#slide=id.p

bajones: want to go back to yesterday's discussion about projection matrices / viewports

<yonet_> PR with a link to the 3DModel Schema definition: https://github.com/google/model-viewer/pull/2853

bajones: new proposal to bridge the differences
… this is a part of layers API since WebGPU is only going to work w/ layers
… we can have both the viewport and projection matrix on XRSubImage
… we end up with 2 projection matrices, one from layers and one from XRView
… we may deprecate the XRView one
… we also have layers that don't have projection matrix, it can be null there
… may be a bit weird but not terrible
… alternative would be to put a new method on the Web[GL/GPU] bindings that returns the matrix
… alternative "solution" is to say that this is a problem and provide the developers with the math that has to happen
… but that is not nice
… what do people think?

cabanier: I prefer optional attribute on XRSubImage

RafaelCintron: thanks for solving, not particularly opinionated, we need to decide what to do when people use the original one
… ok to add a warning for that

bajones: we should be able to add a warning when we see WebGPU being used
… the worst case scenario is if that warning gets ignored is that the site lost half of its depth precision
… mild enough failure that it should be sufficient to just nudge developers

alcooper: any reason to prefer nullable vs returning identity?

bajones: identity is the wrong thing for all the non-projection layers
… 2 main buckets: I have a pre-rendered content that I'm copying onto a layer or I'm actually doing the rendering but it's not aligned with the scene and has its own proj matrix
… there's not really a good matrix that we can hand out

cabanier: agree w/ Brandon
… could we make the warning more aggressive? i.e. maybe remove the attribute if we detect a dev is using WebGPU?

bajones: yes, but that would be a breaking change, but worth considering
… consequences of using the wrong matrix are mild so maybe that's too big stick to use

cabanier: consequences are mild so you won't be able to easily see you're doing the wrong thing
… even though it's a breaking change to the spec, but it won't break any existing content (because you need to take extra steps for this to trigger)

bajones: will send a PR against layers spec
… next thing: retrospective / post-mortem on layers spec
… for the most part seems to be well-aligned, question to Rik: how is the rollout going? any issues?

cabanier: we have not seen too much usage, we're seeing media layers being used
… a-frame supports layers so some samples pop up
… most issues we're seeing are not related to the spec (i.e. lack of multisampling, some multisampling doesn't work with foveation)
… if you use projection layers things are slower compared to vanilla WebXR so focusing efforts there

bajones: hoping to address lack of layers in chrome
… clarification on issues w/ multisampling: is this an issue with something in the driver level?

cabanier: we use GL_EXT_multisampling (?) extension
… if you use that you lose foveation (bad for performance) and we'd like to have both, but that's not currently possible

bajones: let's discuss further; with WebGPU multisampling is something that the user needs to explicitly set up
… there may be some concerns, because if foveation is supposed to happen on a multisampled buffer, we don't have a solution for that

cabanier: porting layers to OpenXR, not finished but working on it

bajones: OpenXR is used for desktop, Chrome and Edge should go through the same path

cabanier: we're hooking into same code, not ready to share yet but hopeful that it'll be possible soon

cabanier: we can chat during weekly call

<yonet_> The OGC GeoPose Standard Working Group (SWG) is the correct name, however, it is not a W3C working group. It is a SWG of the Open Geospatial Consortium, with whom W3C has a liaison.

ada: would be good to discuss what we want to talk about with Geo
… are we interested in it? are we just sitting on it?

<yonet_> https://www.ogc.org/projects/groups/swg

<Leonard> GeoPose spec for comments: https://mail.opengeospatial.org/lists/lt.php?tid=fR5RAlIEAQdSBR1SBQhTSFADVA4ZCQdXBRlfUwUFCwADUQlRXFRKBlIFUVAFXFZIVw9SAhkEBQdXGQBSUlMVBlQBBQdcAQBWBAJRGVALVlJdVgcEGVIJBlQZDQcFUhUPBFsIGVZbBQZTUQkBBwpTBw

ada: any interest?

Leonard: I'm interested, mostly coming from Khronos

<Leonard> Comments on spec - https://github.com/opengeospatial/GeoPose

ada: early WebXR decisions may have been made with the assumption that we may need geopose alignment
… looks like we don't expect any work to be done in this area

cabanier: are we committing to anything?

ada: no, just checking where we are and where they are
… is there anyone with a tiny interest in geoalignment?

yonet: maybe hololens

ada: not comfortable with representing the group's interest in that forum

bialpio: question re persistent / shared anchors - can this be related?

<bajones> Sorry, I have to drop off just a little bit early to pick up kids from school.

<bajones> Great discussion, everyone! Thanks!

ada: not necessarily, mostly about landmarks poses

cabanier: I can ask around if there are people working on something

ada: would be good idea for someone to show up to the meeting and share the status

Leonard: some devices can already support something related with GPS being available

ada: should we remove geo alignment from our charter? no work has happened in 2 years

RafaelCintron: personally no opinion, Alex Turner will have the most context

ada: I'll add a proposal to remove geoaligment from the charter

bialpio: double-check the timezones

Leonard: updated invite coming soon

yonet: minutes should be available in administrivia

Leonard: let's summarize action items on the issues that have been discussed

ada: any objections to skipping the upcoming meeting?
… next one would be Nov 2
… if you need a meeting, email the chairs and ask
… that's it!

<ada> thanks, i almost forgot

<yonet_> Go to bed, thanks

Minutes manually created (not a transcript), formatted by scribe.perl version 158 (Sun Oct 17 00:40:18 2021 UTC).