Immersive Web WG/CG F2F in TPAC, day 2

26 Oct 2018



cwilso, Karen, Sangchul_Ahn, Trevor, johnpallett, cabanier, Yam, chrwilliams, Jungkee_Song, JMedley, svillar, guido_grassel, dom, ada, ChrisLittle, Artem, alexturn, NellWaliczek, LocMDao, Laszlo_Gombos, Dan_Moore, alexismenard, boaz, Simon, Pieters, ddorwin
Chris Wilson, Ada Rose Cannon


<ChrisLittle> present

<dom> ScribeNick: guido_grassel

<dom> ScribeNick: guido_grassel _

CG Overview: review of current incubations

<guido_grassel_> trevor: summary and scope of CG

<guido_grassel_> ... a lot of work going on.

<guido_grassel_> ... follow the repo

<cwilso> trevor: many thanks and appreciation to Jordan Santell and the others contributing to the polyfill

<guido_grassel_> ... proposal repo, what's going

<guido_grassel_> ... pausing and resuming XR session

<guido_grassel_> ... diorama

<dom> Proposals repo: list of ideas as issues

<ada> geo alignment, hit testing and anchor are their own repos. [Add links later]

<inserted> scribenick: guido_grassel__

Chris reminds of visiting other WG

3D Favicons

Rawi: issue #18

<dom> Add support for static 3D favicons #18

Rawi: 3D icon representation: glTF is the right fit

<cwilso> s/rawi/ravi/

Rawi: single contained format

<ada> gltf binary format is suggested i.e. model.glb

Ravi: constraints for GLTF

<ada> We need constraints on 3D favicons size for quick loading.

<ada> Current constraints for images are 144px

small size favicons are preferred, not sure if it should be enforced

scribe: we could just post guidance

<ada> Size should be left up to UA

Chris: should there also be a time axis ?

Ravi: glTF is for 3D.

Rick: we do not want animations
... glTF has animations

Chris: animations around the origin might be ok
... concersation with Magic Leap, HMTL spec does not require a change. just that HTML restricts to images, this is not image.
... HTML spec should not state restrictions on size
... depends on the capability of the device what will work
... use a note in the spec

<Zakim> alexturn, you wanted to ask if the constraints are more about good user experience or protecting against malicious .glb files that DOS the client

<dom> alexturn: in terms of the constraints, they can have a security impact (DOS) vs a UX impact

Ravi: history list should have light weight icons

<ada> DOn't use external textures everything should be self contained

Ravi: GLTF allows external , downloaded textures. make things complex.
... should not use
... the recommendation is based on experiments we have made,

<BrandonJones> For those of us on the phone, what's the content on the screen that's being referred to? Is tehre slides?

<NellWaliczek> https://github.com/immersive-web/proposals/issues/18#issuecomment-416759018

restricttions on shaders

<Zakim> NellWaliczek, you wanted to ask about manifests

<BrandonJones> Thanks

Nell: resource type image vs 3D, also use for Manifest.

Rick: use generically

<ChrisLittle> dse/use/

Neil: What we define should also be useable in Web Manifest

<dom> Icons in Web App Manifest

<Zakim> ada, you wanted to ask whether we want to constrain the bounding box

Ada: Maifest allows icons for different purposes, like home screen, and favicon ...


scribe: author might prefer a diarama , like app icons in Daydream home screen
... for representation as a Web app
... what should be the bounding box. - one unit cube, would there be cropping, scaling

<alexturn> Some limits imposed on glTF models for the Windows MR home screen: https://docs.microsoft.com/en-us/windows/mixed-reality/creating-3d-models-for-use-in-the-windows-mixed-reality-home

Ravi: we do not want to crop anything, scaling is possible.
... should be left to the UA

<Zakim> alexismenard, you wanted to ask about if animated 3D favicons can become a UX/usability for example blinking high frequency

alexismenard: two uses cases favicon, app manifest: they have different constraints.
... we should split the use cases
... do we want animations?
... animations of many icons might lead to bad UX

<Zakim> cwilso, you wanted to comment about animations

<ada> magicleap animate on interaction/hover

Chris: built-in animations might not be supported , its up to the UA.
... but a UA might animate a selected icon
... suggested bounds, to give a hint how big icon should be

Ravi: UA scales to make them all same

Do people want to force an icon to look smaller....

<Blair_Moz> dom, thanks, sorry about that, everyone.

BrandonJones: animations, supported in some placed (not everywhere): first frame pose needs to be clear.
... otherwise non-anim icons will look strange.
... size of an icon in meters

<ada> It's amazing what artists can do with size constraints.

<ada> Lowpoly art 😻

BrandonJones: limit should be number of triangles, textures
... motivation: these are icons, preview not full representation

ada: Playstation 2 gives good example of icons we want to achieve here, simplified representation of game characters

<Zakim> NellWaliczek, you wanted to discuss negotiation of correct asset

<ada> reference: Icons on the PS2 memory card

Nell: order of icons in the Web Manifest strict semantics.

<ada> https://i.imgur.com/Pwbwsp8.jpg

<trevorfsmith> CSS already has cm and mm units!

Nell: physically smaller instead of digitally smaller.

ChrisW, Nell, constructed physical size!

<trevorfsmith> "q, mm, cm, in: Quarter millimeters, millimeters, centimeters, or inches."

scribe: icn size selection by smaller constructed physical size

Branon: high quality or low quality

<ada> 10cm: bust.glb, 50cm: waist-up.glb, 100cm: full model.glb

Nell: brand guidelines when to use what icon

Ravi: glTF allows to multiply vertices , bypasses size restrictions
... what is the dimension for the 3rd dimension

Nell: give hnts when to use what content
... low / high quality is good
... optimized for viewing at 1meter

Bandon: high / low detail sounds useful, may not correlate with the complexity of the model,

<dom> [it does feel like we need to amend HTML/Web App Manifest for 3D assets]

ada: what the model shows might be different depending on the detail that can be shown.

<Zakim> cwilso, you wanted to ask about restrictions enforcement

cwilso: what if I build a wall that is sometimes 5m away , sometimes 10m away, do icons change?

<BrandonJones> +q

Nell: content author wants give guidelines based on the phy size of the bounding box.

<dom> Icons could be auditory icons, visual icons, or other kinds of icons.

BrandonJones: we want to define which side of the model should be shown by default,

<dom> also from the same HTML spec section: "There is no default type for resources given by the icon keyword. However, for the purposes of determining the type of the resource, user agents must expect the resource to be an image."

BrandonJones: rotate full 360 instead of just slight turning.

<dom> (which has an impact I think for content type sniffing)

BrandonJones: icon might be a full model or a shell.

<dom> https://html.spec.whatwg.org/multipage/semantics.html#concept-link-type-sniffing

Next: Web Audio .....

Ravi: Action items:

DOM: Manifest and HMTL amendment

Ravi: We need functionality that allowsthe author participate content selection

<dkrowe> something to consider here - in the same way that artists might want to use different models at different display sizes, they may also want different materials in different lighting conditions

ChrisW: continue this discussion to proposals repo.

<dkrowe> The PBR restriction on GLTF makes this easier because PBR is more consistent than other lighting models, but artists might still say “if I knew this was going to be back-lit I would’ve emphasized the edges more”

ChrisW: guidelines how to make a icon are needed
... changes in Web Manifest might not happen in this group, but drafting something

Ada: what we draft in proposal repo might also graduate elsewhere.

<ada> The WebAudio people havejust arrived 🎶

<cwilso> ACTION: Trevor to create a repo for the guidance to authors

Dom: ... do push this into the WG, one might need to join the Web Platform WG for IP reasons.

Joint meeting with Web Audio

<dom> scribenick: johnpallett

<ada> Our hero!! ⛑

cwilso: when we drafted IWWG one of the main threads of feedback was about audio

From web audio team: Paul Adenot (editor of Web Audio API, Firefox/Mozilla)

scribe: Hongchan (editor, Chrome)
... Takashi Toyoshima (Chrome, working on Midi)
... Matthew Paradis (co-chair of Web Audio WG; works for BBC)

(note: Raymond Toy is other chair and couldn't make it)

scribe: Varun Nair (Facebook, web audio, recently joined web audio WG)
... Stephan Letz
... Michel Buffa (Universite de Cote Azure)
... Nigel Megitt (BBC, observer)

cwilso: So how does audio fit into immersive web? Current WebXR gives access to displays and positional devices, creates immersive environment by linking to rendering contexts, etc. Haven't put audio specifically into that, but we have what we need for this and the THREE.js library already does this - it has a panner node which it synchronizes with the head pose, so if you're using with WebXR in the future it will position with sound correctly.
... there is an issue open requesting that things be easier, i.e. not having to synchronize yourself by watching XR and synchronizing with pose/frames

Paul: We recently added feature to the web audio API allowing you to chose speed at which particular audio param should render at. We sample at 60, 120, 240Hz which is lower than the audio rate but still somewhat high. May be possible to clock to the display rate, seems like consensus of scientific community.

ada: to control pitch or quality?

paul: no, it reduces processing load - e.g. can sample every 128 audio samples instead of every audio sample which allows optimizations

cwilso: audio processing happens much faster than video processing - 44,100 samples per second, you tend to batch them, how much batching is done is allowed to change
... batching reduces processing requirements but increases latency since each batch has to be processed before playback

general perception research says 80-100ms is where you can perceive lag in the audio field vs. video field. You want head tracking to be processed as quickly as possible.

varun: said the bit above about research

cwilso: your head position may have a lower latency requirement than if you're physically doing something in an immersive world that you'd expect to make sound; if it takes too long to make sound you may not connect it. E.g. if you press a key you expect a click in ~10ms range
... e.g. musicians will notice lag otherwise.
... a lot of web audio substructure was built for RTC which originally had ~150ms latency requirements... this was no good for musicians

Hongchan: Association with PannerNode is a good idea but what's missing is that in Web Audio we don't have an element for ambisonic audio; we had to build something on top of Web Audio (e.g. resonance SDK)
... hopefully that can be addressed collaboratively as well

ada: what's ambisonic audio?

<cwilso> ACTION: add proposal issue to track ambisonic audio

varun: conceptually it's a panoramic map of audio - spatial information can be compressed into a few channels/tracks to get intensity of different audio signals in 3D
... it's both a storage format and a rendering format
... used for 360 videos and immersive media
... easier to stream N channels of audio instead of 1000s of individual sounds with 3D positions
... lots of companies have been implementing renderers, would be great if this could be embedded with some ability to customize

cwilso: ambisonic audio is recorded from a fixed point right? you can't move around, you can rotate?

varun: yup

cwilso: may be interesting to align properly, is it a 360 video use case?

varun: interactive scene - may have lots of content that you've rendered this way. A lot of sound is rendered this way
... used a lot for ambient sounds, it's a performance win to do that instead of having all ambient audio baked as individual sound effects

brandon: can you comment on resonance library?

hongchan: omnitone (which is the basic component of ambisonic built on top of the web audio API) - the resonance SDK went a step further...

<dom> Omnitone, Spatial Audio Rendering on the web

<dom> Omnitone is a JavaScript implementation of an ambisonic decoder that also allows you to binaurally render an ambisonic recording directly on the browser.

hongchan: ambisonic field is usually fairly static. But resonance SDK added a sense of virtual space based on ambisonic technology by adding echos, spatial processing

<dom> With Resonance Audio, bring dynamic spatial sound into your VR, AR, gaming, or video experiences at scale.

hongchan: working on v2 of Web Audio API at TPAC. Working out how to adopt spatial audio more naturally in the API
... don't have a clear idea yet, but we're working on it so it's good timing for joint conversations

cwilso: also working on device management, correct? Currently use the default audio device for web audio?

varun: yup. on agenda for 2
... v2.

ada: do you imagine ambisonic audio will be part of web audio API?

hongchan: not sure. trying to work out spatial audio more generally. ambisonic is only one spatialization technique; Panner node is another.
... there are various ways to implement audio spatialization; want to incorporate all of them in a nice and concise way

cwilso: ambisonics is like 360 video in this way; could be a backdrop to the scene, could place things in the scene but it's not the only thing

ada: any more questions?
... thanks everyone for visiting!

<ChrisLittle> * thank you johnpallett for immpressive scribing

DOM overlays

<dom> Slides: Presenting 2D Web Content in XR [PDF]

<dom> Administrativia repo

nell: to recap decision at last f2f, there are AR security implications of working inline
... particularly on smartphones people might think that you'd just embed it inline
... but there are security issues with that. So, immersive-only for AR is the current plan.
... there is issues with that though - if you restrict content to be immersive-only it forces all user interfaces to be built in GL
... and it's a high barrier, and seems like we lose the advantages of having all this nice HTML and web technologies

<dom> Notes from September 17th AR Face-to-Face of Immersive Web CG

nell: so, DOM overlays came up... and it's complex. So Jill will cover investigations that we've done to solve this problem.

[ed: I'll go into more detail briefly on security issues in privacy section later]

<dom> [slide 1]

Jill: Presenting findings that were the result of exploring ways to present 2D DOM content in an XR context

nell: Note that this is CG work - an investigation into an approach to solve the problem, it needs more work, staying in CG until design gets stable enough.

<dom> [slide 2]

<dom> [slide 3: Background: 2D Content in XR]

Jill: [slide 3] 2 different problems here - how do you display DOM elements on the screen is different than DOM in the world.

<dom> [slide 4: Background - Compositors, Quad Layers]

Jill: definition of compositor: During render pipeline a compositor creates different layers for rendering, and when done rendering in separate processes, it applies them on top of each other. Allows more efficient rendering, also allows projective error fixing, can be done without requiring entire frame to be rendered.
... typically these compositor systems expose different layers of different types.

definition of quad layer: It optimizes rendering of elements in pixel space in a 3D scene.

Jill: We thought quad layers might be a good way to address 2D DOM content in a scene

Nell: A lot of headset platforms need to use similar techniques to allow last-second adjustment of pose right before final submission of frame to the headset. i.e. in the milliseconds it takes to do drawing, user might not be where the headset thought
... some techniques move pixels or even blur them when moving things around. This prevents crisp and clear text if it's been merged directly into the main texture that's being reprojected
... so quad layers allow you to separate text as a separate texture and allows it to be composited with different rules that have a better efficacy to preserve legibility off text
... there's an awesome demo of this. Try it if you can.

Jill: So... how do we expose these two concepts to the browser?

<dom> [slide 5: User Agents and Compositor Layers: CSS3D]

<max> scribenick: max

jill: css styles - reserve a layer for this element so we can do optimizations like smoothing and other rendering optimizations for this use-case

<dom> [Slide 6: Background - DOM to Texture]

nell: some of this might be user-agent specific in terms of implementation specifics

jill: we don't have a way of putting dom directly into a rendering context, so devs usually fake this with dom-to-texture
... this is not a new problem. It's something we have been doing since webGL was introduced
... but how do we safely render dom to texture? no way to do that without breaking iframe and cross-origin content
... no way to do this without exposing the x-origin content pixel data and preserve the separate browser context
... we want to discourage devs from doing this

<dom> [Slide 7: Options we investigated]

nell: there were experiments with this, but nobody has shipped due to security concerns. For example, theme data from the OS could be exposed. We are aware and sensitive to these issues as we look at options moving forward

jill: with that in mind, these are the options we explored for exposing DOM in XR context
... first option - do nothing - tell devs we don't support this. Not a great option
... 2 option - could we expose a DOM overlay layer or controls in the immersive mode but hide content in other contexts

nell: big concern with that is compatibility

jill: could either take the 2d DOM overlay for the immersive experience and manage it at the UA level and find a way to represent it in world space for the use
... the developer would create this for immersive and the agent would pick how to represent it in world space
... or use a quad layer - separate rendering context but then the dev can position it
... the agent-positioned option is not an ideal user experience

nell: we decided to walk away from quad layers for now due to issues at the last f2f. These overlays on screen space - we can't know what the intent behind these overlays would be.
... it might be buttons, but it might be more fancy
... it might be a thing in 2d space tracking something in 3d space
... if we translate that automatically - problems! - how large should it be? There are lots of complex problems here
... user interface aspects are usually left up to the UA
... at least with the UA-driven approach, devs who didn't test their UI on the headset at least get something working

<dom> [Slide 8: Option 2 - DOM to Screen]

<ddorwin> One option would be to require the application to indicate the intended use of the overlay so that headset-based UAs can make intelligent decisions in how/where to render it.

nell: what we are going to talk about for the rest of the time is look at the design we are targeting - rendering 2D content in DOM-to-screen representation

jill: concerns with this problem space is we need to render this 2d content - get it into the render tree. Should it be parented to the DOM heirarchy?
... one of the things we explored is the fullscreen API
... elements themselves can request fullscreen - takes that tree rendering and presents that element as a solo fullscreen experience. Could we do something similar in this space for 2d controls as an overlay over the immersive AR presentation
... give that this is an overlay, could it be its own quad layer?

nell: another thing to add about fullscreen - where it differs from fullscreen, with XR you can have a headset plugged into a PC and still have main browser window on desktop screen
... if the element is in the tree - is it being drawn on the main desktop browser window at the same time it is engaged in the headset? Lots of questions here.

<dom> [Slide 9: Option 4 - DOM to World]

jill: if we were to implement this as a quad layer under the hood for the end user, what would it take to extend this to DOM-to-world
... do we accept input now that we are in world-space or is static content only sufficient
... some platforms do have static only
... if we do have it interactive, how does that work? Lots of questions here.
... not all underlying platforms even support a quad layer - problems with things like composition order, occlusion, and input

<dom> [Slide 10: Inputs]

jill: how do we possibly add this layer through the WebXR API and what would have to change to provide that functionality?
... in order to interact with the DOM content, the UA would have to inject inputs into these elements by providing the intersection data through the raycast or input controller and tranlsate those to 2d coordinates in windows space

nell: no matter what you still ahve to render the world. You have to know where your head, your input sources, regardless of whether you are injecting input into a quad in 2d space, that input still has to go to the underlying 3d world

<dom> q° guido_grassel__ to introduce virtual desktop use case

nell: still have to know where in 3d space the quad is - what its physical dimensions. The UA has to compute the x,y hit on that quad
... it would fire those events on the quad like it was a 2d page
... the same data the UA would use to compute that has to be available to the page that is drawing the broader 3d content

<dom> [Slide 11: Cross-Origin DOM Content]

nell: both get the same data.. problems!

jill: fine for things in the same content/page/origin, but what about x-origin?
... two potential feature sets - static vs. interactive
... just getting the visual representation, is there a security concern there?
... even though we are reserving pixel space, we aren't exposing the content to the parent rendering context
... has security parity with what we have today
... but issue with input is not
... exposing the same input data to the parent browsing content and the 2d x-origin content
... some of the DOM-to-texture stuff comes back into play, like click-jacking
... as an attacker I can render a page with a "click here to win an iPhone" but hide your bank account button behind it

<johnpallett> +q to elaborate on the x-origin input threat vector

jill: in the XR context if you position a transparent object that collects that input as well as pass it through to the x-origin content, that attack vector is viable again
... maybe x-origin content is just visual representation
... we would need a signal from the origin that this content is valid to be presented in an immersive context

<dom> [Slide 12: Differences in underlying platform support - Platform Capabilities]

jill: but that could work to make it non-interactive
... the third concern is quad layer support on underlying platform - whether it supported at all and feature parity questions
... high level summary: there is not feature parity among platforms that support quad layers

<dom> [the cross-origin input delegation feels more like a mission for Feature Policy than CSP if I got this right]

jill: maybe the UA could abstract these differences and shim this across the platforms

<dom> [Slide 13: Next Steps]

jill: given that is our investigation, what would it take to move forward? what do we still need to answer before a formal proposal?
... first, we need to rationalize a common abstraction for a quad-layer API across platforms even without feature parity. What would be the minimum set to expose to user
... what chnages would be necessary to the webXR API
... the webXR session only supports base-layer. What would that change look like and what would we expose?
... if we did provide support, would this be a one-time change or would it inspire other features to expose more layers
... it seems like there is just the two - 3d rendering and quad layer. Some confidence those two are sufficient in this space <audible groans>
... hope is we can identify collaborators to question these assumptions and gather this into a strong proposal
... questions?

<BrandonJones> +q

<Zakim> alexturn, you wanted to talk about leaning towards a design that allows UAs to experiment with option 3 as a fallback, in case something is better than nothing (similar to Desktop

alexturner: two things. one - for option 3, we don't need to explicitly pursue this but maybe just tweak the option 2 design to not get in the way of UAs that want to experiment
... for half the sites it might be horrible, but similar to desktop mode on phones, it might work for sites that don't work otherwise
... two - if we don't go all the way to generic quad layer API, if we just have dom-to-screen and we just have one screen overlay layer with a targeted slot like that - you can make one layer and if you add an extra attribute that pushes it in to the world. Maybe that's sufficient and then later we can do a fully generic quad layer API

nell: the same security issues for x-origin would apply there as well. Most of our investigations aren't prescriptive of API shape. We do want to do the full investigation to see what the full API with no restrictions to one-panel would be.
... one of the things we are worried about is feature creep keeps us from shipping anything.
... we want to make sure we think through this enough to see what the full thing would be before we start cutting corners off

<Zakim> ddorwin, you wanted to talk about option 3 and requiring the application to indicate the intended use of the overlay so that headset-based UAs can make intelligent decisions in

david: follow-up - there would be value in the simple thing. How do we attach the DOM? could iterate on that etc.
... related to option 3 - could require the app to give hints - like "this is a hud" or "just controls" so put that on a hand or whatever and then the UA can experiment more

nell: those aren't mutually-exclusive - providing those hints doesn't rule out providing an explicit quad-layer API later
... might be a documentation challenge, but one doesn't inherently rule out the other

david: window in front of you, left bar, where do you put controls in your platform? Hints could be very useful to UA

nell: is this actually less complex than a generic quad layer system? Once you get into enumerating all these options, it might be more complex for UAs to implement these hint systems

<Zakim> guido_grassel__, you wanted to introduce virtual desktop use case

guido: I am concerned about feature creep here
... there can be more complex use-cases - we all know these virtual desktop cases where I have a bunch of pages in a virtual desktop situation and can move them all around and change the stacking order etc
... might sound far-fetched but in chrome canary, this is implementable
... you can use css-3d and iframes to place different frames with correct perspective based on camera position and set them around in front of you
... that used to be possible recently - we may want to consider how far we want to take this
... all those iframes were fully interactable - you coudl type in input fields, etc, all that stuff really worked

nell: one of the biggest feature requests we get at Sumerian is this works great on handheld but fails completely on handheld devices. They build experiences that they are happy with and then try them in VR and oh no! <womp womp>
... one of the big questions we get - why can't I use the tools I know to build this stuff? Well - on the headset, this stuff just often doesn't work!
... these core concepts of using 2d on the web sometimes don't work and it can be very hard for developers

<Blair_Moz> should point out that HTML and WebGL content doesn't mix in 3D so it creates problems

nell: yeah you can use CSS 3d systems, but it doesn't often work well in headsets

ada: do you think it's worth finding ways to get the browser to render iframes stereoscopically?

<Blair_Moz> the depth buffer is the problem, too

nell: it's not something we looked into - it doesn't really solve all the underlying issues with compositor etc. We want to get this stuff in the hands of developers.

<Zakim> johnpallett, you wanted to elaborate on the x-origin input threat vector

<alexturn> Blair_Moz: I think the CSS 3D transforms gudio_grassel__ mentioned were used to position each 2D slate itself in 3D, rather than to pop things out of the slates

johnpallett: followup on click jacking - want to cover threat vector about x-origin DOM content where the top level page is getting pose info and then input in the iframe - the top-level page could sniff input based on pose data that is being inputted

<Blair_Moz> ?

johnpallett: you can figure out what someone is typing into the iframe page based on accelerometer data etc

nell: yeah, this x-origin content problably can't take input at all unless these origins are marked explicitly as taking content from each other

brandonjones: I think your response to John answered part of my question - wanted to say you can't x-origin input in XR. Input is my major concern besides feature creep
... would be possible for a dev to shift the page under the cursor to make the user click on an ad. It would be possible to put a page in the scene and then render something in 3d over the top so the user doesn't even know they are interacting with a 3d page.
... if we are restricting input to a single origin that is better

nell: when we investigated this this week we discovered we can do something similar today in the 2d web - not to say we just have to be equally secure but it does already exist - an element can cover x-origin content but let the input go in even thought he x-origin frame isn't even visible

iank: we have been working on platform-level APIs to solve this problem. Intersection APIs V2 - there are potential solutions to this space

nell: that wasn't ultimately what made us walk away from this - if you are rendering this stuff in VR, there are a lot of input sources
... head pose can be used for gaze-select
... controller can be used for point and select

<dom> IntersectionObserver v2

nell: tablet or phone can be used as a tap from screen space
... main page must receive this input or it can't render
... if you want to make sure that the input goes into the page but can't be used by the scene, the scene can't render properly anymore
... we tried restricting just to pointer pose but you need to know the entire geo of the scene
... <in-person 3d input prototpye demo>
... peculiar situation where only the underlying page could know a hit occurred due to obscure occlusion scenario
... need to dig into these issues more
... including issues of composition - layer ordering and accuracy of input - still digging into a lot of these issues

<Zakim> alexturn, you wanted to ask about <100% immersive coverage for screen-space control panel on the side, per Blair_Moz's feedback at the AR F2F

<mmocny> For those remote, here's a visual: https://lh3.googleusercontent.com/fzhkTeAssxTuWukPrdHTb-MKGAYTWb6VFtx2WkgzsIoDasBJOj1LCOZxsT5jFQOxWYMpPiiYtrxncKWasD7ck1HkYHGvJ3Ipa-WgZBXhcVMUa2tRup1C-X7z28-ZMv6rXx-QBB7NK5w=w1011-h758-no

artem: occulus definitely interested in implementing this in VR (not AR)
... for the text presentation in VR, the cylinder layer is much better than quad

it's much more comfortable to have something curved in front of you rather than flat

scribe: [11:58] <ddorwin> q+ to note that there are other types of "input" to the page. For example, intersection observer. Thus, we either need to disallow such APIs when a frame is used this way or ensure we are providing accurate data (and considering all the eye tracking, etc. concerns). Also, what does it mean for a pixel to be visible in a
... I hope we are talking about adding layers to webXR
... why limit to just one
... cylinder layer, cube map, etc
... it's more comfortable to have cylinder than flat UI

nell: not every platform supports quad layers. To have compatibility across the web, the burden would fall on the UA to do this
... how do we define this feature in a scoped way to be implementable in the time frames we are looking at
... what is the smallest viable feature set that enables this stuff?
... when we start looking at cylinder layers, etc, it starts to blow up in time frame to implement on all UAs
... just a quad layer seems like it might be reasonable but we still have to investigate even that
... we are looking for the smallest viable feature set to start with

<Zakim> alexturn, you wanted to ask about <100% immersive coverage for screen-space control panel on the side, per Blair_Moz's feedback at the AR F2F

alexturner: for the screen space stuff, Blair mentioned the way some apps use screen space today to have control panels with like a shifted immersive view with a fully opaque screen-space view
... does the immersive view have to take up the full view or can we have UI on part of it

nell: we haven't really thought about the implications on camera feed etc. in the AR context specifically - like not preserving the aspect ratio
... I suspect (completely unconsidered thought, straight off the dome) - question of opacity or not - could build a screen space overlay that blocked the camera feed or one that compressed the feed
... nothing in our designs today that would enable compressing to a subsection of the screen
... if I recall from last convo with Blair, he said as long as we can do hit-testing from any point on the screen that we don't need to be able to compress into a subsection

blair: Yeah, the only weird thing is sometimes people would create simple AR apps where people would create AR apps with the content in the center of the view frustum - like auto-add some furniture
... if half the screen is covered, then the center of the view is kind of half-clipped
... the 3d math would do things in surpri9sing ways to people but it's all dealable

<Zakim> ddorwin, you wanted to note that there are other types of "input" to the page. For example, intersection observer. Thus, we either need to disallow such APIs when a frame is used

david: with any kind of live DOM, what does it mean to be visible in terms of occlusion, etc

nell: yeah, is it an element with a subtree, is it its own Document, is it an iframe? Still noodling, just wanted to plant the seed to get feedback on these topics from the group
... opening the conversation, feedback wanted!

<trevorfsmith> It's 4am and I'm going to sleep. 👋

<NellWaliczek> sleep well!!

<dom> Guido's slides on multitasking

<dom> Expose multitasking support for webXR #15

<boaz> scribe: boaz

<dom> scribenick: boaz

<dom> [slide 1: Multitasking]


<dom> [slide 2: AR Scene +3D objects]

Guido: I read the proposal #15 on multitasking and thought it was interesting and wanted to talk about use cases

<Blair_Moz> slides?

I have two use cases to present for AR Scenes and 3 opbjects; virtual vacuum robot controls and virtual advertising

[link to slides above]

the main point is the javascript used to program the a/r scene and the javascript used to drive the robot.

the a/r scene author may not have anticipated the robot, and it may be a cross origin negotiation

the second example is that there may be objects in the world that are adverts and are third party javascript programs

<dom> [slide 3: What we already have]

advances slides

guido: the way I see it today is that there would be a main thread , and the 3rd party javascript from the robot and the advertisement would run in one thread with my a/r scene
... of course I can spawn workers in the background
... two issues: one the thread may have too much to do and drop frames
... the second is, there may be xss issues

<dom> [slide 4: Solution ideas]

guido: what I want to know is where we figure this out
... one solution i want to discuss is; is it possible to have the xfr send objects to multiple contexts

we could call these layers and run them in different contexts and separate them from each other for the purposes of securityu

<Zakim> alexturn, you wanted to talk about how immediate composition has multidraw concerns, and how retained platforms built on top of webxr could allow multiple sites to submit geometry

alex: this is an interesting thing going on on native platforms, too: how do we make a platform where different authors' programs can interact.
... also we are seeing this in secondlife

<Blair_Moz> +q to summarize argon experience

alex: it feels like a lot of our immediate mode apis are stable, but there is this really rich exploration going on in a retained mode, where people are walking around and their spheres are interacting and they are running multiple authors' content
... so one way we can think about this rich time is how do we provide concepts to authors for dealing with this
... is it DOM?

<ddorwin> s/alex;/alex:/

alex: how do we prevent people from using native to get around sec primitives.
... so that they use the web.

mmocny: i think there is def value in thinking about layers for multiple js contexts, but since you refered to the preso I gave yesterday
... my proposal yesterday was that authors should present content statically as data so that users can see it privately and performantly
... because simply pointing your headset at the world does not represent user intent for interacting with a third party

<Zakim> Blair_Moz, you wanted to summarize argon experience

alex: some of those people are going to aggregate content

Blair_Moz: the comment from google points to the need for something like this.
... the only way to do this in a non google website is to run your own app.

<dom> argon.js - A javascript framework for adding augmented reality content to web applications

<boazsender> nell: it doesn't sound like we've reached enough consensus to establish next steps.

<dom> scribenick: dom

WebXR & Testing

Boaz: I'm Boaz Senders, from Bocoup, a consulting team working with browser vendors on WPT
... we work on developing test, maintain the CI for running the tests
... my presence here is out of curiosity about your approach to testing

Nell: some background on existing discussions
... started last year in TPAC but stalled out

<ddorwin> WebXR Test API repo/explainer: https://github.com/immersive-web/webxr-test-api/blob/master/explainer.md

Nell: two sides to the coin:
... how do UA test themselves? how do XR content creator test their apps?
... we had been hoping to have something that would work for both
... I had conversations with James Graham on extending Web Driver for this
... that direction has value
... we sketched what functions might be necessary
... a number of the considerations are independent of WebXR per se, but are linked to hardware behavior

<Zakim> jillian_munson, you wanted to provide context on the sumerian use case for browser testing, what features we need

Jill: for Sumerian specifically, I can talk about what we would like browsers to expose
... for testing automation
... we would love to have one automation platform to drive both browser and device events without dealing with hardware
... mocking the browser and headset behaviors
... testing targeted at the WebXR device API is one thing, but we also want to get capture of the generated images, performance aspects

boaz: that makes sense
... we have a proposal from one our coworker to create device mocks

nell: for what kind of hardware?

boaz: this was contemplated for the case of getUserMedia
... but it could be generalized to other hardware and sensors

jill: would it make sense to do that with the generic sensor API?

boaz: that's what I had assumed we would be doing for VR/AR, but it looks like XR is at a higher level than these low-level sensors
... Do you have a sense of your timeline compared to generic sensor?

Nell: the sensor that powers XR are not exposed as generic sensors
... there may be common patterns which we could replicate
... our charter brings us to CR by end of Q2 2019

<Zakim> ddorwin, you wanted to mention WebXR Test API and our use in Blink.

ddorwin: there is a repo for the testing XR API

<NellWaliczek> link please?

ddorwin: it's specific to WebXR, but it's not one-to-one to the API
... it could be made more generic


scribe: we're using this for testing chrome
... it's js based before wrapping it in mojo
... could be built in web driver
... we could iterate on the JS side pretty quickly before baking stuff in Web Driver

zcorpan: can you elaborate on the testing approach in Blink? Is this mocks or actual devices?

DavidD: we would want to go deeper into our testing, but there are issues, e.g. security
... another option would be to expose openVR in some way
... a lot of this is implementation specific, and may be difficult to get running in wpt.fyi

boaz: for wpt.fyi, one of the things that we do is run the CI infrastructure that collects the results
... on sauce, AWS, on our own servers
... we also running nightly versions of browsers
... so we could run tests on non-production chrome

zcorpan: you could also expose things via command-line flags, where it would be disabled by default
... e.g. to disable some security constraints

Nell: one of the challenges when trying to build middleware on the Web for all these devices
... there are fundamentally behavior from device to device
... one of the things we have to do at Amazon is to validate across browsers and across devices
... it becomes a disaster of a test matrix and thus reduce our commitment to where the platform is going to work
... we have to encapsulate some of these different hardware behaviors in some way to simplify this process

boaz: this may take the form of providing this encapsulation to hardware manufacturers

Nell: is there any learning from OpenXR in this space?

Walk through issues marked for short updates

<boazsender> thanks everyone for having us!

<boazsender> :wave:

Issues labeled as FTF discussion requested

Default FoV for magic window canvases #272

Brandon: the question is whether we just pick a value as a default? is this something we leave to developers?
... or should we assume that in abssence of head tracking the user face is .5m from the screen?

<Zakim> Blair_Moz, you wanted to related some experience

Blair_Moz: we did research around this - users tend to move their head to get closer to the screen leading to bad results

Brandon: have you used zSpace devices?

Blair: not recently

Brandon: they have a system place where leaning toward the screen produces a very natural result on the 3D content

Nell: propose to keep that as part of the continuation of the issue comments

<ada> https://github.com/immersive-web/webxr/issues/203

Expose combined frustum from both eyes #203

<Blair_Moz> brandon: I think most zspace content (that I saw) used stereo and sat above the display; in our case, we used a handheld and had the content behind the display

Nell: the discussion from yesterday could have an impact on that

<Blair_Moz> this probably had a big effect.

Nell: on multiview rendering
... I propose we wait to get the outcome of that discussion before picking this again

design "Lite AR Mode" that doesn't require permissions/consent equivalent to camera #29

Blair: I've posted a request for feedback on some of the ideas on the thread - got some (but not a lot of) feedback

<Zakim> Blair_Moz, you wanted to talk about simple AR mode

Blair: at a high level, the idea is to define approaches for e.G. hit testing that can fail
... there is a massive spectrum of possibilities which would need user testing
... please bring feedback

Nell: maybe a good one for the CG teleconf in 10 days
... we'll stop now given our meeting with Web Platform WG now

Ada: we're going to floor -2 - we will be discussing Gamepad API
... come and join us if you're interested

<daoshengmu> test

<BrandonJones> testack

<daoshengmu> Will we discuss Advanced WebXR Controller Input #392 in TPAC?

<johnpallett> Reminder: Privacy group will arrive at 3pm

<alexturn> scribenick: alexturn

Declarative models

cabanier: Here from Magic Leap, startup in Florida
... device is similar to HoloLens, but has a device in your pocket to power the device
... Created LuminOS as a custom OS here
... also wanted to let folks place things in the environment
... immersive applications and landscape applications
... immersive is like WebXR: draw to frame buffer every frame
... landscape applications are ilke on the screen, where they all work together and display together
... persistent - they come back when you turn off and restart your device
... users don't think of them as applications - just something always there
... apps run in bounded areas called prisms
... prisms constrain the app to operate within a bounding box
... within each prism, there is a scene graph made of nodes
... nodes can models/images/sounds/transform nodes
... scene graph is dynamic and animatable - can be built and then changed on fly
... apps are sandboxed and can't see each other's scene graphs
... apps each live in their own process and communicate with the universe
... it's like a compositor that takes data from all landscape applications over ipc and tells apps when user looks at app or away from app
... browser shows previews of sites and apps with little 3d icon on top
... want to give JS devs access to this environment too
... wayfair site: web page displays on front
... has prism behind the site which JS from site can manipulate
... for JS, all content is relative to browser window - no idea of where user is or world is
... some people wanted custom HTML tags, but decided against that
... instead, created custom elements and low-level JS APIs to call into Lumin runtime
... Chris created Prismatic, which devs can include on HTML web pages
... can then do <ml-model> or <ml-quad> and decide where they're placed in environment
... constrain prisms on where they can be placed - if trying to place farther away, ask user for permission
... remembers per origin if user allows use of entire environment
... Prismatic.js calls into Lumin engine, calling Helio and then LuminRT

<ml-model> has style and src tags and allows leaper model to appear on page and leap out

scribe: <ml-model> has style and src tags and allows leaper model to appear on page and leap out
... can also allow users to click on chair and extract it into environment

cwilso: Can I grab anything to make something draggable into the environment?

chrwilliams: Can add a tag to an element on the page to make it draggable into environment

<ada> Sorry I opened the queue

<ada> cwilso: can you make sure you use a mic for questions

cabanier: Chose this model because it's very simple
... If people run out of power with ml-model, they can go to low-level API, but we discourage it

<ravi> yes, one page can create more than one prism

cabanier: what do people think? Did we do the right thing here?
... Using Prismatic lets us switch out the library as we talk further as a group

<Zakim> Blair_Moz, you wanted to comment on gltf ...

Blair_Moz: I like this sort of thing in general - been thinking about how to enable multitasking
... may have web pages you're not even looking at that add critters to your environment
... would love to look further
... in the past, another issue raised is how to ensure that any given glTF looks the same on each platform
... depends on the renderer
... don't personally think this matters that much as long as glTF is generally a standard

<ada> +1 to tht things can look different on other platforms

Blair_Moz: curious how much back and forth on what you allow
... can I dynamically change a texture continuously as the prism runs

cabanier: yes, that's certainly an option

<ada> The power of declarative is allowing the platform to adopt content for the appropriate audience

cabanier: no limits, can change at any time
... can apply color/transparency

NellWaliczek: People might presume that since I'm so focused on an imperative API, that I'm not in favor of declarative
... that's not true, this is a big part of where we need to go, and is why our CG is called the Immersive Web CG
... there's a whole family of specs that will make up what folks think of as their options
... for immersive web, folks will have many reasons to pick among these options
... don't think you don't have my support here

cabanier: yea, that's why we pursue the custom element model

NellWaliczek: yea, for now as people think about model formats, that may already be somewhat well defined
... as we start to get to multitasking or cross-origin content, now the complexity that comes in can block people
... accessibility! i could keep going
... when thinking of the problem space as what can the UA provide that would simply not be available if it doesn't open up an API, that's where we are most leveraged to make a sea change on the web
... part of why i'm here in the immersive web group is to make this kind of thing happen!

(switching to the privacy interest group's visit)

Joint meeting with Privacy Interest Group

<dom> IWCG Privacy Update [PDF]

ada: Chris and Ada are the WG chairs, Nell and Brandon are the spec editors, and Trevor is the CG chair

johnpallett: I'm John, we have a privacy-and-security repo, and I've been managing issues in that
... could easily fill 2 hours, and we've got 30 minutes

Tara: Co-chair of privacy group, working at Google

Jason: Working at Apple

<dom> Immersive Web Privacy and Security

johnpallett: First 15m are to do a super-fast overview of privacy for our group, both for privacy team and those in our group who are not focused yet on privacy
... deck is long and we won't fully go through it but you can read the deck online
... still more detail in the repo in the explainer
... want to open things up for bidirectional discussion, and so we'll then open this up to you
... WebXR is an API to let apps build VR and AR experiences
... from VR where everything is virtual to AR where you add content to the world
... API gives developers device poses for rendering and gives real-world data for augmenting the world

<dom> [slide What is WebXR]

johnpallett: on an AR display, the UA manages the camera composition, but you get to render your own content

<dom> [slide Privacy Approach]

johnpallett: also supports see-through displays, where you see the real world optically
... our privacy approach is to identify XR-specific threat vectors
... want to highlight some key observations our group has had

Jason: Are you tracking threat vectors in one doc/issues?

johnpallett: Tracking as issues for now, and will move into explainer

<dom> [slide Threat vectors]

johnpallett: Threat vectors: 1) *Perception* of camera access

<dom> [Slide Why does it matter? [perception of Camera access]

johnpallett: In AR, the user may see the camera rendered and believe the site can access the camera, even if we did prevent pixel access

<dom> [slide Perception of Camera Access]

johnpallett: Even without camera access, site could pop up a screen to appear like it can capture picture and blackmail user
... Even on see-through devices, app could show a record indicator and pretend like it's recording, even if device doesn't have camera

<dom> [Slide Real-World Geometry Data]

johnpallett: Mitigation: Could communicate to user that site doesn't have access
... to camera

<dom> [Slide 10 Why does it matter?]

2) Real-world Geometry Data

scribe: 2) Real-world Geometry Data

<scribe> ... new APIs give apps access to real-world shape of the user's room

<dom> [slide 11 Threat Vectors]

UNKNOWN_SPEAKER: good because it lets apps lock objects at precise places in real world
... threats: apps can get PII: face analysis, embossed credit card #

<dom> [slide 12 Threat Vectors]

UNKNOWN_SPEAKER: could fingerprint using room geometry
... locate user even without permission
... could profile income/height/previous visited places/bounding boxes/etc.

<dom> [Slide 13 Potential Mitigations]

UNKNOWN_SPEAKER: could access historical data from previous sessions where there may be ghosts of previous room planes/data that the user had cleared away
... Mitigations: throttling/precision doesn't really mitigate problems here, only hurting the experience
... low-res data might be visualized by the user - how do we tell the user what's going on?
... can we ask the user somehow whether to just do low-res data?

<dom> [Slide 14 Object Identification]

UNKNOWN_SPEAKER: opportunities for AR subsystem to filter the scope of data - perhaps just new data for new session

<dom> [Slide 15 Why does it matter?]

UNKNOWN_SPEAKER: 3) Object Identification

<dom> [Slide 16 Threat Vectors (Object Identification)]

UNKNOWN_SPEAKER: in AR, good to find known objects to annotate them
... problem: user can't easily understand what's being scanned for
... could find location through storefronts/qr code/street sign/configuration

<dom> [slide 17 Object Identification POssible mitigations]

<dom> [slide 18 Permissions]

mitigations: for keeping users safe (prevent blocking stop sign), could perhaps limit access to precise position?

<dom> [back to slide 16]

Jason: In the first threat vector, sounded like they couldn't get camera, but now they can?

johnpallett: Site could perhaps ask user agent "tell me when you see cup" and that could give data to tapp
... or could build an api that uses camera to find poster, still without actual camera access

Jason: So should we presume apps don't have access to camera?

johnpallett: Don't have a conclusive answer to whether apps will get camera access

NellWaliczek: Currently shaped to not give camera data

Thanks to John for doing great work on due diligence here

scribe: Thanks to John for doing great work on due diligence here

<dom> [slide 18 Permissions]

scribe: some of this is actually further out than it might seem

johnpallett: 4) Permissions

<dom> [slide 19 Permissions]

johnpallett: this kind of wraps up a lot of the topics we covered
... worried about permission fatigue
... can i have camera? world data? water bottles? say yes to everything
... users may not understand all these implications
... other question is around scope/duration of permissions
... may be easy to understand wanting to enter an AR experience, but may not expect that to be permanent
... exploring idea of an AR mode
... will be immersive not inline so that it's explicitly permissioned
... have explicit session scope to enter/exit with user intent

<dom> [I'll note that this "session mode" is similar to what is done with screen sharing in webrtc]

johnpallett: also gives UA more ability to show custom UI

NellWaliczek: Should explain immersive in context of handheld device

johnpallett: Like full-screen: a mode you enter and exit
... on headset display, would render the scene without rendering anything else

<dom> [slide 20 Pose and Frame of Reference Data]

NellWaliczek: Dom mentioned it's similar to screen-sharing in WebRTC

johnpallett: Powering through in remaining 11 minutes

<dom> [slide 21 Access to Pose and Frame of Reference Data]

johnpallett: In order to render 3D geometry, device position and orientation are necessary to provide
... Knowing how high off of floor and space bounds within which user can move
... Local frame of reference - doesn't by itself give real-world location
... could be on phone/laptop/headset

<dom> [slide 22 Threat Vectors]

johnpallett: trying not to box us in based on form factor

NellWaliczek: 2D browsers as well

johnpallett: Headset displays also can show 2D browsers in 3D environment
... that web page could then use the WebXR API
... to show a 3D element within a larger 2D space
... other threats:
... Gaze tracking to know what the user is looking at
... Should pages be able to know what you look at outside the page?
... Device orientation will allow you to infer what keystrokes are being entered
... Could read PINs on phones with orientation at 20hz
... could affect gaze on virtual keyboards

<dom> [slide 23 WebXR Modes, Implied Threats]

johnpallett: location profiling: can determine location within city with 400m of trajectory trace

<dom> [Slide 24 Potential Mitigations]

johnpallett: some frames of reference restrict what data is available
... input sniffing is tough: a lot of mitigations look similar to generic sensor data
... allow data only to focused and visible page

<dom> [slide 25 … more generally...]

<dom> [slide 26 Potential Mitigations]

johnpallett: if sitting in chair, may be able to limit bounds of pose data
... other mitigations for gaze tracking such as blurring things

<dom> [slide 29 Join the conversation!]

johnpallett: please do join the conversation if you're not part of this yet!

<Zakim> NellWaliczek, you wanted to define gaze tracking

NellWaliczek: gaze tracking and head pose tracking may not be the same
... may be different threats with eye vs. head direction

johnpallett: Privacy group: Sorry for firehose! Do you have any Qs?

Tara: Thanks! Blown away by sheer amount of threats but you've done so much work on mitigations!

<dom> Self-Review Questionnaire: Security and Privacy

Tara: Looking at similar things with sensor API, and would love to know how you got to this exhaustive list.

johnpallett: Haven't yet found big book of XR threats
... some existing research into these areas
... may be areas we need to research ourselves, such as user prompts

Tara: Snickering because we have the same item on our agenda
... you may be one step ahead of our own agenda

johnpallett: Ideas on how we can best engage with you?

Tara: We have our own github with privacy questionnaire - a lot of things take place as conversations
... do teleconferences once a month, can review materials ahead of time and give you an hour to bang on issues
... given amount of issues here, would make sense to have a longer conversation

ada: Almost out of time...

weiler: Where are you on hard mitigations such as not putting data into the API or not making it available?

johnpallett: In terms of our process, we're doing it in three steps: 1) threat vectors 2) potential mitigation 3) specific language

NellWaliczek: Could take a stance and decide not to give data, but you can, in essence, get everything from camera
... could generally be safer to encourage use of non-camera streams for most data streams

johnpallett: One idea of a hard mitigation is inline vs immersive for frames of reference

NellWaliczek: Not sure what you're referring to here?

johnpallett: There are multiple frames of reference a view can use that result in different levels of data access
... some only available when in immersive mode

NellWaliczek: Still have valid reasons for override option - for diorama mode
... so may not be as simple to secure inline mode

<Zakim> NellWaliczek, you wanted to discuss future collboration

johnpallett: Would like to know how we can best help you

Tara: Don't have a super strict method to impose - don't worry too much about that

weiler: Like the holistic threats that emerge when you look at the whole picture

Jason: Holisitic is best to start for those of us new to XR

NellWaliczek: To help get context, may be best to do quick demos now before future calls

<ravi> Thanks John Pallett

<jillian_munson> scribenick: jillian_munson

Declarative Models strike back

cwilso: declarative models, had an offline conversation with ML team
... 1. Momentum behind gltf, glb being a baseline interchange format
... Feel that this is not an industry consensus, wanted to ask the room for thoughts

Rik: Yes, this is most requested format from developers.

<Artem> +1

<max> +1

<NellWaliczek> +1

<alexturn> +1

<ravi> +1

<cwilso> +1

<alexismenard> +1

<lgombos> +1

<dom> +1

<cabanier> +1

<Blair_Moz> +1

<ada> +1

<chrwilliams> +1

<arno_> +1

<DanMoore> +1

<Blair_Moz> note: we'll have to deal with versions

<alexturn> +2.0

<ravi> version 2.0

<ada> 💘

nell: distinction, only talking about gltf 2.0

<Blair_Moz> As long as it's defined, "any version" will do

cwilso: Yes, also in favor of declarative solutions. Focusing on basics of WebXR, this is not necessarily where we are going to land, open to declarative interface

<dom> The Extensible Web Manifesto

cwilso: Important to figure out core functionality of API to build on top of
... Comment on Magic Leap implementation: assumes that there is an initial pose, orientation for rendering
... May need to think about doing this in context without that position data

<Blair_Moz> Nell, sure. The point is that as long as people using it know "2.0" or "2.1+" or whatever, and models created with 1.1 are identifiable as such, it's "fine"

cwilso: wanted us to start thinking about these use cases for models, where we should be incubating them. If not a proposal, at least a proof of concept to start the conversation
... how we should incubate this

<alexturn> I think glTF 1.0 and 2.0 have the same MIME type, so it could be dangerous if some browsers start randomly supporting glTF 1.0 and we all end up compelled to do so, since it's not really easy to secure

cwilso: but in the short term, focusing on the device API

ada: Really excited about what the Magic Leap team has developed so far

<johnpallett> +q to say that we should also document existing declarative 3D model approaches in addition to use cases

max: Agree with Nell, this is a huge topic to unpack
... Our team has also been exploring the declarative space, thinking more broadly on how to deconstruct pages into elements in world space
... interesting features - progressive enhancements for pages that can transition between experiences, experiences with 2D elements embedded
... very interested in this, great start to the conversation

<dom> LensVR The VR/AR web browser

max: Would love to set up a presentation with folks not here from team to demo for this group

<Zakim> NellWaliczek, you wanted to discuss apple's involvement

NellWaliczek: lots of designs around things that try to be declarative systems, flavors of declarative design
... some confusion in the ecosystem today, would love if we could proactively reach out to our developers

<ravi> Max is this the one you were talking about: http://www.joshcarpenter.ca/declarative-3d/

<Zakim> johnpallett, you wanted to say that we should also document existing declarative 3D model approaches in addition to use cases

NellWaliczek: run the risk of fragmentation, we should set up a conversation with Apple to discuss what's the same, what's different, get consensus

johnpallett: Would be good to capture some use cases, different declarative approaches that have been implemented
... beyond just use cases today

<johnpallett> johnpallett: wants us to list what's been tried before, capabilities, pros and cons so that we can learn from past experiments

alexturn: Example, threejs, a-frame also declarative-type models even if they are driven entirely in javascript space
... should consider these use cases and what they have done as well

<johnpallett> +1 thanks Alex

mmochny: Assuming declarative, browser understands the content, etc.
... Is the model handled by the page? Pose data?

rik: No, once it gets generated the model belongs to the system not the page
... can close the page, model persists and belong to the user space

mmocny: With respect to pose data, the page does not need to know how that is done?

rik: correct

<Zakim> NellWaliczek, you wanted to talk about khronos

rik: Specifically used custom elements, if community comes up with something else we are willing to switch out underlying implementation

NellWaliczek: one thing to be aware of, gltf 2.0 is being driven by khronos
... if there are things that are inadequate, changes we want to request, logistical complexities
... there are folks in this group that also members of khronos to facilitate
... have a model of how we communicate across standards body, need to be sensitive

alexturn: Microsoft is a member of khronos, requested extensions to gltf
... could we request an extension if we are not part of the group?

NellWaliczek: My guess would be no

<dom> glTF Extension Registry

NellWaliczek: Similar conversation yesterday around WebGL and support functionality we need

<dom> "To create a new extension, use the extension template and open a pull request into this repo. Make sure to add the extension to the glTF Extension Registry (top of this file)."

NellWaliczek: we can use that to figure out how those conversations should happen
... Brandon is in both groups, will follow up with him

<dom> "Extensions start as a vendor extension, then can become a multi-vendor extensions if there are multiple implementations, and can become a ratified Khronos extension (the multi-vendor extension is an optional step)."

ada: Any last comments for this topic?

<max> thanks to Dom for doing all the random IRC wrangling

<alexturn> +30


cwilso: Wanted to get a sense of whether or not we could change the meeting time to be on the hour and not on the half hour

ada: If you have the existing meeting invite, please delete

<DanMoore> +!

<Blair_Moz> on the hour is fine for me

<DanMoore> +1

<gen> +1

cwilso: Wanted to get a sense of whether or not this was a shared problem

<alexturn> +60

<ravi> +1

<NellWaliczek> +1

<Blair_Moz> +1

<chrwilliams> +1

<johnpallett> +1

<max> +1

cwilso: +1 for on the hour time zone

<alexismenard> +1

<Artem> +0

<alexturn> +90

<Blair_Moz> We should do a doodle poll on +/- 30 minutes

cwilso: expect a doodle to change the time for the meeting

<gen> earlier please ...

<cwilso> ACTION: chris to send out a doodle poll to change time.

ada: we will do the poll, sorry if you are unable to make it
... tuesday's meeting is a working group meeting
... agenda has gone out

<dom> [regrets for Tuesday from me as well]

NellWaliczek: Won't be there, on vacation

ada: if you want to add descriptions for agenda topics, please open a pull request
... has all call details, tuesday at 6:30 UK time

dom: Also, there is daylight savings...

ada: Meeting is pinned to west coast time

cwilso: also, there is a F2F in january from 1/29-1/30

ada: It will be in Samsung America
... and now you are all free

Summary of Action Items

[NEW] ACTION: add proposal issue to track ambisonic audio
[NEW] ACTION: chris to send out a doodle poll to change time.
[NEW] ACTION: Trevor to create a repo for the guidance to authors

Summary of Resolutions

[End of minutes]

Minutes manually created (not a transcript), formatted by David Booth's scribe.perl version 1.154 (CVS log)
$Date: 2018/11/06 09:12:57 $