<ChrisLittle> present
<dom> ScribeNick: guido_grassel
<dom> ScribeNick: guido_grassel _
<guido_grassel_> trevor: summary and scope of CG
<guido_grassel_> ... a lot of work going on.
<guido_grassel_> ... follow the repo
<cwilso> trevor: many thanks and appreciation to Jordan Santell and the others contributing to the polyfill
<guido_grassel_> ... proposal repo, what's going
<guido_grassel_> ... pausing and resuming XR session
<guido_grassel_> ... diorama
<dom> Proposals repo: list of ideas as issues
<ada> geo alignment, hit testing and anchor are their own repos. [Add links later]
<inserted> scribenick: guido_grassel__
Chris reminds of visiting other WG
Rawi: issue #18
<dom> Add support for static 3D favicons #18
Rawi: 3D icon representation: glTF is the right fit
<cwilso> s/rawi/ravi/
Rawi: single contained format
<ada> gltf binary format is suggested i.e. model.glb
Ravi: constraints for GLTF
<ada> We need constraints on 3D favicons size for quick loading.
<ada> Current constraints for images are 144px
small size favicons are preferred, not sure if it should be enforced
scribe: we could just post guidance
<ada> Size should be left up to UA
Chris: should there also be a time axis ?
Ravi: glTF is for 3D.
Rick: we do not want
animations
... glTF has animations
Chris: animations around the
origin might be ok
... concersation with Magic Leap, HMTL spec does not require a
change. just that HTML restricts to images, this is not
image.
... HTML spec should not state restrictions on size
... depends on the capability of the device what will
work
... use a note in the spec
<Zakim> alexturn, you wanted to ask if the constraints are more about good user experience or protecting against malicious .glb files that DOS the client
<dom> alexturn: in terms of the constraints, they can have a security impact (DOS) vs a UX impact
Ravi: history list should have light weight icons
<ada> DOn't use external textures everything should be self contained
Ravi: GLTF allows external ,
downloaded textures. make things complex.
... should not use
... the recommendation is based on experiments we have
made,
<BrandonJones> For those of us on the phone, what's the content on the screen that's being referred to? Is tehre slides?
<NellWaliczek> https://github.com/immersive-web/proposals/issues/18#issuecomment-416759018
restricttions on shaders
<Zakim> NellWaliczek, you wanted to ask about manifests
<BrandonJones> Thanks
Nell: resource type image vs 3D, also use for Manifest.
Rick: use generically
<ChrisLittle> dse/use/
Neil: What we define should also be useable in Web Manifest
<dom> Icons in Web App Manifest
<Zakim> ada, you wanted to ask whether we want to constrain the bounding box
Ada: Maifest allows icons for different purposes, like home screen, and favicon ...
s/maifest/Manifest
scribe: author might prefer a
diarama , like app icons in Daydream home screen
... for representation as a Web app
... what should be the bounding box. - one unit cube, would
there be cropping, scaling
<alexturn> Some limits imposed on glTF models for the Windows MR home screen: https://docs.microsoft.com/en-us/windows/mixed-reality/creating-3d-models-for-use-in-the-windows-mixed-reality-home
Ravi: we do not want to crop
anything, scaling is possible.
... should be left to the UA
<Zakim> alexismenard, you wanted to ask about if animated 3D favicons can become a UX/usability for example blinking high frequency
alexismenard: two uses cases
favicon, app manifest: they have different constraints.
... we should split the use cases
... do we want animations?
... animations of many icons might lead to bad UX
<Zakim> cwilso, you wanted to comment about animations
<ada> magicleap animate on interaction/hover
Chris: built-in animations might
not be supported , its up to the UA.
... but a UA might animate a selected icon
... suggested bounds, to give a hint how big icon should be
Ravi: UA scales to make them all same
Do people want to force an icon to look smaller....
<Blair_Moz> dom, thanks, sorry about that, everyone.
BrandonJones: animations,
supported in some placed (not everywhere): first frame pose
needs to be clear.
... otherwise non-anim icons will look strange.
... size of an icon in meters
<ada> It's amazing what artists can do with size constraints.
<ada> Lowpoly art 😻
BrandonJones: limit should be
number of triangles, textures
... motivation: these are icons, preview not full
representation
ada: Playstation 2 gives good example of icons we want to achieve here, simplified representation of game characters
<Zakim> NellWaliczek, you wanted to discuss negotiation of correct asset
<ada> reference: Icons on the PS2 memory card
Nell: order of icons in the Web Manifest strict semantics.
<ada> https://i.imgur.com/Pwbwsp8.jpg
<trevorfsmith> CSS already has cm and mm units!
Nell: physically smaller instead of digitally smaller.
ChrisW, Nell, constructed physical size!
<trevorfsmith> "q, mm, cm, in: Quarter millimeters, millimeters, centimeters, or inches."
scribe: icn size selection by smaller constructed physical size
Branon: high quality or low quality
<ada> 10cm: bust.glb, 50cm: waist-up.glb, 100cm: full model.glb
Nell: brand guidelines when to use what icon
Ravi: glTF allows to multiply
vertices , bypasses size restrictions
... what is the dimension for the 3rd dimension
Nell: give hnts when to use what
content
... low / high quality is good
... optimized for viewing at 1meter
Bandon: high / low detail sounds useful, may not correlate with the complexity of the model,
<dom> [it does feel like we need to amend HTML/Web App Manifest for 3D assets]
ada: what the model shows might be different depending on the detail that can be shown.
<Zakim> cwilso, you wanted to ask about restrictions enforcement
cwilso: what if I build a wall that is sometimes 5m away , sometimes 10m away, do icons change?
<BrandonJones> +q
Nell: content author wants give guidelines based on the phy size of the bounding box.
<dom> Icons could be auditory icons, visual icons, or other kinds of icons.
BrandonJones: we want to define which side of the model should be shown by default,
<dom> also from the same HTML spec section: "There is no default type for resources given by the icon keyword. However, for the purposes of determining the type of the resource, user agents must expect the resource to be an image."
BrandonJones: rotate full 360 instead of just slight turning.
<dom> (which has an impact I think for content type sniffing)
BrandonJones: icon might be a full model or a shell.
<dom> https://html.spec.whatwg.org/multipage/semantics.html#concept-link-type-sniffing
Next: Web Audio .....
Ravi: Action items:
DOM: Manifest and HMTL amendment
Ravi: We need functionality that allowsthe author participate content selection
<dkrowe> something to consider here - in the same way that artists might want to use different models at different display sizes, they may also want different materials in different lighting conditions
ChrisW: continue this discussion to proposals repo.
<dkrowe> The PBR restriction on GLTF makes this easier because PBR is more consistent than other lighting models, but artists might still say “if I knew this was going to be back-lit I would’ve emphasized the edges more”
ChrisW: guidelines how to make a
icon are needed
... changes in Web Manifest might not happen in this group, but
drafting something
Ada: what we draft in proposal repo might also graduate elsewhere.
<ada> The WebAudio people havejust arrived 🎶
<cwilso> ACTION: Trevor to create a repo for the guidance to authors
Dom: ... do push this into the WG, one might need to join the Web Platform WG for IP reasons.
<dom> scribenick: johnpallett
<ada> Our hero!! ⛑
cwilso: when we drafted IWWG one of the main threads of feedback was about audio
From web audio team: Paul Adenot (editor of Web Audio API, Firefox/Mozilla)
scribe: Hongchan (editor,
Chrome)
... Takashi Toyoshima (Chrome, working on Midi)
... Matthew Paradis (co-chair of Web Audio WG; works for
BBC)
(note: Raymond Toy is other chair and couldn't make it)
scribe: Varun Nair (Facebook, web
audio, recently joined web audio WG)
... Stephan Letz
... Michel Buffa (Universite de Cote Azure)
... Nigel Megitt (BBC, observer)
cwilso: So how does audio fit
into immersive web? Current WebXR gives access to displays and
positional devices, creates immersive environment by linking to
rendering contexts, etc. Haven't put audio specifically into
that, but we have what we need for this and the THREE.js
library already does this - it has a panner node which it
synchronizes with the head pose, so if you're using with WebXR
in the future it will position with sound correctly.
... there is an issue open requesting that things be easier,
i.e. not having to synchronize yourself by watching XR and
synchronizing with pose/frames
Paul: We recently added feature to the web audio API allowing you to chose speed at which particular audio param should render at. We sample at 60, 120, 240Hz which is lower than the audio rate but still somewhat high. May be possible to clock to the display rate, seems like consensus of scientific community.
ada: to control pitch or quality?
paul: no, it reduces processing load - e.g. can sample every 128 audio samples instead of every audio sample which allows optimizations
cwilso: audio processing happens
much faster than video processing - 44,100 samples per second,
you tend to batch them, how much batching is done is allowed to
change
... batching reduces processing requirements but increases
latency since each batch has to be processed before
playback
general perception research says 80-100ms is where you can perceive lag in the audio field vs. video field. You want head tracking to be processed as quickly as possible.
varun: said the bit above about research
cwilso: your head position may
have a lower latency requirement than if you're physically
doing something in an immersive world that you'd expect to make
sound; if it takes too long to make sound you may not connect
it. E.g. if you press a key you expect a click in ~10ms
range
... e.g. musicians will notice lag otherwise.
... a lot of web audio substructure was built for RTC which
originally had ~150ms latency requirements... this was no good
for musicians
Hongchan: Association with
PannerNode is a good idea but what's missing is that in Web
Audio we don't have an element for ambisonic audio; we had to
build something on top of Web Audio (e.g. resonance SDK)
... hopefully that can be addressed collaboratively as well
ada: what's ambisonic audio?
<cwilso> ACTION: add proposal issue to track ambisonic audio
varun: conceptually it's a
panoramic map of audio - spatial information can be compressed
into a few channels/tracks to get intensity of different audio
signals in 3D
... it's both a storage format and a rendering format
... used for 360 videos and immersive media
... easier to stream N channels of audio instead of 1000s of
individual sounds with 3D positions
... lots of companies have been implementing renderers, would
be great if this could be embedded with some ability to
customize
cwilso: ambisonic audio is recorded from a fixed point right? you can't move around, you can rotate?
varun: yup
cwilso: may be interesting to align properly, is it a 360 video use case?
varun: interactive scene - may
have lots of content that you've rendered this way. A lot of
sound is rendered this way
... used a lot for ambient sounds, it's a performance win to do
that instead of having all ambient audio baked as individual
sound effects
brandon: can you comment on resonance library?
hongchan: omnitone (which is the basic component of ambisonic built on top of the web audio API) - the resonance SDK went a step further...
<dom> Omnitone, Spatial Audio Rendering on the web
hongchan: ambisonic field is usually fairly static. But resonance SDK added a sense of virtual space based on ambisonic technology by adding echos, spatial processing
hongchan: working on v2 of Web
Audio API at TPAC. Working out how to adopt spatial audio more
naturally in the API
... don't have a clear idea yet, but we're working on it so
it's good timing for joint conversations
cwilso: also working on device management, correct? Currently use the default audio device for web audio?
varun: yup. on agenda for 2
... v2.
ada: do you imagine ambisonic audio will be part of web audio API?
hongchan: not sure. trying to
work out spatial audio more generally. ambisonic is only one
spatialization technique; Panner node is another.
... there are various ways to implement audio spatialization;
want to incorporate all of them in a nice and concise way
cwilso: ambisonics is like 360 video in this way; could be a backdrop to the scene, could place things in the scene but it's not the only thing
ada: any more questions?
...
... thanks everyone for visiting!
<ChrisLittle> * thank you johnpallett for immpressive scribing
<dom> Slides: Presenting 2D Web Content in XR [PDF]
<dom> Administrativia repo
nell: to recap decision at last
f2f, there are AR security implications of working inline
... particularly on smartphones people might think that you'd
just embed it inline
... but there are security issues with that. So, immersive-only
for AR is the current plan.
... there is issues with that though - if you restrict content
to be immersive-only it forces all user interfaces to be built
in GL
... and it's a high barrier, and seems like we lose the
advantages of having all this nice HTML and web
technologies
<dom> Notes from September 17th AR Face-to-Face of Immersive Web CG
nell: so, DOM overlays came up... and it's complex. So Jill will cover investigations that we've done to solve this problem.
[ed: I'll go into more detail briefly on security issues in privacy section later]
<dom> [slide 1]
Jill: Presenting findings that were the result of exploring ways to present 2D DOM content in an XR context
nell: Note that this is CG work - an investigation into an approach to solve the problem, it needs more work, staying in CG until design gets stable enough.
<dom> [slide 2]
<dom> [slide 3: Background: 2D Content in XR]
Jill: [slide 3] 2 different problems here - how do you display DOM elements on the screen is different than DOM in the world.
<dom> [slide 4: Background - Compositors, Quad Layers]
Jill: definition of compositor:
During render pipeline a compositor creates different layers
for rendering, and when done rendering in separate processes,
it applies them on top of each other. Allows more efficient
rendering, also allows projective error fixing, can be done
without requiring entire frame to be rendered.
... typically these compositor systems expose different layers
of different types.
definition of quad layer: It optimizes rendering of elements in pixel space in a 3D scene.
Jill: We thought quad layers might be a good way to address 2D DOM content in a scene
Nell: A lot of headset platforms
need to use similar techniques to allow last-second adjustment
of pose right before final submission of frame to the headset.
i.e. in the milliseconds it takes to do drawing, user might not
be where the headset thought
... some techniques move pixels or even blur them when moving
things around. This prevents crisp and clear text if it's been
merged directly into the main texture that's being
reprojected
... so quad layers allow you to separate text as a separate
texture and allows it to be composited with different rules
that have a better efficacy to preserve legibility off
text
... there's an awesome demo of this. Try it if you can.
Jill: So... how do we expose these two concepts to the browser?
<dom> [slide 5: User Agents and Compositor Layers: CSS3D]
<max> scribenick: max
jill: css styles - reserve a layer for this element so we can do optimizations like smoothing and other rendering optimizations for this use-case
<dom> [Slide 6: Background - DOM to Texture]
nell: some of this might be user-agent specific in terms of implementation specifics
jill: we don't have a way of
putting dom directly into a rendering context, so devs usually
fake this with dom-to-texture
... this is not a new problem. It's something we have been
doing since webGL was introduced
... but how do we safely render dom to texture? no way to do
that without breaking iframe and cross-origin content
... no way to do this without exposing the x-origin content
pixel data and preserve the separate browser context
... we want to discourage devs from doing this
<dom> [Slide 7: Options we investigated]
nell: there were experiments with this, but nobody has shipped due to security concerns. For example, theme data from the OS could be exposed. We are aware and sensitive to these issues as we look at options moving forward
jill: with that in mind, these
are the options we explored for exposing DOM in XR
context
... first option - do nothing - tell devs we don't support
this. Not a great option
... 2 option - could we expose a DOM overlay layer or controls
in the immersive mode but hide content in other contexts
nell: big concern with that is compatibility
jill: could either take the 2d
DOM overlay for the immersive experience and manage it at the
UA level and find a way to represent it in world space for the
use
... the developer would create this for immersive and the agent
would pick how to represent it in world space
... or use a quad layer - separate rendering context but then
the dev can position it
... the agent-positioned option is not an ideal user
experience
nell: we decided to walk away
from quad layers for now due to issues at the last f2f. These
overlays on screen space - we can't know what the intent behind
these overlays would be.
... it might be buttons, but it might be more fancy
... it might be a thing in 2d space tracking something in 3d
space
... if we translate that automatically - problems! - how large
should it be? There are lots of complex problems here
... user interface aspects are usually left up to the UA
... at least with the UA-driven approach, devs who didn't test
their UI on the headset at least get something working
<dom> [Slide 8: Option 2 - DOM to Screen]
<ddorwin> One option would be to require the application to indicate the intended use of the overlay so that headset-based UAs can make intelligent decisions in how/where to render it.
nell: what we are going to talk about for the rest of the time is look at the design we are targeting - rendering 2D content in DOM-to-screen representation
jill: concerns with this problem
space is we need to render this 2d content - get it into the
render tree. Should it be parented to the DOM heirarchy?
... one of the things we explored is the fullscreen API
... elements themselves can request fullscreen - takes that
tree rendering and presents that element as a solo fullscreen
experience. Could we do something similar in this space for 2d
controls as an overlay over the immersive AR presentation
... give that this is an overlay, could it be its own quad
layer?
nell: another thing to add about
fullscreen - where it differs from fullscreen, with XR you can
have a headset plugged into a PC and still have main browser
window on desktop screen
... if the element is in the tree - is it being drawn on the
main desktop browser window at the same time it is engaged in
the headset? Lots of questions here.
<dom> [Slide 9: Option 4 - DOM to World]
jill: if we were to implement
this as a quad layer under the hood for the end user, what
would it take to extend this to DOM-to-world
... do we accept input now that we are in world-space or is
static content only sufficient
... some platforms do have static only
... if we do have it interactive, how does that work? Lots of
questions here.
... not all underlying platforms even support a quad layer -
problems with things like composition order, occlusion, and
input
<dom> [Slide 10: Inputs]
jill: how do we possibly add this
layer through the WebXR API and what would have to change to
provide that functionality?
... in order to interact with the DOM content, the UA would
have to inject inputs into these elements by providing the
intersection data through the raycast or input controller and
tranlsate those to 2d coordinates in windows space
nell: no matter what you still ahve to render the world. You have to know where your head, your input sources, regardless of whether you are injecting input into a quad in 2d space, that input still has to go to the underlying 3d world
<dom> q° guido_grassel__ to introduce virtual desktop use case
nell: still have to know where in
3d space the quad is - what its physical dimensions. The UA has
to compute the x,y hit on that quad
... it would fire those events on the quad like it was a 2d
page
... the same data the UA would use to compute that has to be
available to the page that is drawing the broader 3d
content
<dom> [Slide 11: Cross-Origin DOM Content]
nell: both get the same data.. problems!
jill: fine for things in the same
content/page/origin, but what about x-origin?
... two potential feature sets - static vs. interactive
... just getting the visual representation, is there a security
concern there?
... even though we are reserving pixel space, we aren't
exposing the content to the parent rendering context
... has security parity with what we have today
... but issue with input is not
... exposing the same input data to the parent browsing content
and the 2d x-origin content
... some of the DOM-to-texture stuff comes back into play, like
click-jacking
... as an attacker I can render a page with a "click here to
win an iPhone" but hide your bank account button behind it
<johnpallett> +q to elaborate on the x-origin input threat vector
jill: in the XR context if you
position a transparent object that collects that input as well
as pass it through to the x-origin content, that attack vector
is viable again
... maybe x-origin content is just visual representation
... we would need a signal from the origin that this content is
valid to be presented in an immersive context
<dom> [Slide 12: Differences in underlying platform support - Platform Capabilities]
jill: but that could work to make
it non-interactive
... the third concern is quad layer support on underlying
platform - whether it supported at all and feature parity
questions
... high level summary: there is not feature parity among
platforms that support quad layers
<dom> [the cross-origin input delegation feels more like a mission for Feature Policy than CSP if I got this right]
jill: maybe the UA could abstract these differences and shim this across the platforms
<dom> [Slide 13: Next Steps]
jill: given that is our
investigation, what would it take to move forward? what do we
still need to answer before a formal proposal?
... first, we need to rationalize a common abstraction for a
quad-layer API across platforms even without feature parity.
What would be the minimum set to expose to user
... what chnages would be necessary to the webXR API
... the webXR session only supports base-layer. What would that
change look like and what would we expose?
... if we did provide support, would this be a one-time change
or would it inspire other features to expose more layers
... it seems like there is just the two - 3d rendering and quad
layer. Some confidence those two are sufficient in this space
<audible groans>
... hope is we can identify collaborators to question these
assumptions and gather this into a strong proposal
... questions?
<BrandonJones> +q
<Zakim> alexturn, you wanted to talk about leaning towards a design that allows UAs to experiment with option 3 as a fallback, in case something is better than nothing (similar to Desktop
alexturner: two things. one - for
option 3, we don't need to explicitly pursue this but maybe
just tweak the option 2 design to not get in the way of UAs
that want to experiment
... for half the sites it might be horrible, but similar to
desktop mode on phones, it might work for sites that don't work
otherwise
... two - if we don't go all the way to generic quad layer API,
if we just have dom-to-screen and we just have one screen
overlay layer with a targeted slot like that - you can make one
layer and if you add an extra attribute that pushes it in to
the world. Maybe that's sufficient and then later we can do a
fully generic quad layer API
nell: the same security issues
for x-origin would apply there as well. Most of our
investigations aren't prescriptive of API shape. We do want to
do the full investigation to see what the full API with no
restrictions to one-panel would be.
... one of the things we are worried about is feature creep
keeps us from shipping anything.
... we want to make sure we think through this enough to see
what the full thing would be before we start cutting corners
off
<Zakim> ddorwin, you wanted to talk about option 3 and requiring the application to indicate the intended use of the overlay so that headset-based UAs can make intelligent decisions in
david: follow-up - there would be
value in the simple thing. How do we attach the DOM? could
iterate on that etc.
... related to option 3 - could require the app to give hints -
like "this is a hud" or "just controls" so put that on a hand
or whatever and then the UA can experiment more
nell: those aren't
mutually-exclusive - providing those hints doesn't rule out
providing an explicit quad-layer API later
... might be a documentation challenge, but one doesn't
inherently rule out the other
david: window in front of you, left bar, where do you put controls in your platform? Hints could be very useful to UA
nell: is this actually less complex than a generic quad layer system? Once you get into enumerating all these options, it might be more complex for UAs to implement these hint systems
<Zakim> guido_grassel__, you wanted to introduce virtual desktop use case
guido: I am concerned about
feature creep here
... there can be more complex use-cases - we all know these
virtual desktop cases where I have a bunch of pages in a
virtual desktop situation and can move them all around and
change the stacking order etc
... might sound far-fetched but in chrome canary, this is
implementable
... you can use css-3d and iframes to place different frames
with correct perspective based on camera position and set them
around in front of you
... that used to be possible recently - we may want to consider
how far we want to take this
... all those iframes were fully interactable - you coudl type
in input fields, etc, all that stuff really worked
nell: one of the biggest feature
requests we get at Sumerian is this works great on handheld but
fails completely on handheld devices. They build experiences
that they are happy with and then try them in VR and oh no!
<womp womp>
... one of the big questions we get - why can't I use the tools
I know to build this stuff? Well - on the headset, this stuff
just often doesn't work!
... these core concepts of using 2d on the web sometimes don't
work and it can be very hard for developers
<Blair_Moz> should point out that HTML and WebGL content doesn't mix in 3D so it creates problems
nell: yeah you can use CSS 3d systems, but it doesn't often work well in headsets
ada: do you think it's worth finding ways to get the browser to render iframes stereoscopically?
<Blair_Moz> the depth buffer is the problem, too
nell: it's not something we looked into - it doesn't really solve all the underlying issues with compositor etc. We want to get this stuff in the hands of developers.
<Zakim> johnpallett, you wanted to elaborate on the x-origin input threat vector
<alexturn> Blair_Moz: I think the CSS 3D transforms gudio_grassel__ mentioned were used to position each 2D slate itself in 3D, rather than to pop things out of the slates
johnpallett: followup on click jacking - want to cover threat vector about x-origin DOM content where the top level page is getting pose info and then input in the iframe - the top-level page could sniff input based on pose data that is being inputted
<Blair_Moz> ?
johnpallett: you can figure out what someone is typing into the iframe page based on accelerometer data etc
nell: yeah, this x-origin content problably can't take input at all unless these origins are marked explicitly as taking content from each other
brandonjones: I think your
response to John answered part of my question - wanted to say
you can't x-origin input in XR. Input is my major concern
besides feature creep
... would be possible for a dev to shift the page under the
cursor to make the user click on an ad. It would be possible to
put a page in the scene and then render something in 3d over
the top so the user doesn't even know they are interacting with
a 3d page.
... if we are restricting input to a single origin that is
better
nell: when we investigated this this week we discovered we can do something similar today in the 2d web - not to say we just have to be equally secure but it does already exist - an element can cover x-origin content but let the input go in even thought he x-origin frame isn't even visible
iank: we have been working on platform-level APIs to solve this problem. Intersection APIs V2 - there are potential solutions to this space
nell: that wasn't ultimately what
made us walk away from this - if you are rendering this stuff
in VR, there are a lot of input sources
... head pose can be used for gaze-select
... controller can be used for point and select
<dom> IntersectionObserver v2
nell: tablet or phone can be used
as a tap from screen space
... main page must receive this input or it can't render
... if you want to make sure that the input goes into the page
but can't be used by the scene, the scene can't render properly
anymore
... we tried restricting just to pointer pose but you need to
know the entire geo of the scene
... <in-person 3d input prototpye demo>
... peculiar situation where only the underlying page could
know a hit occurred due to obscure occlusion scenario
... need to dig into these issues more
... including issues of composition - layer ordering and
accuracy of input - still digging into a lot of these
issues
<Zakim> alexturn, you wanted to ask about <100% immersive coverage for screen-space control panel on the side, per Blair_Moz's feedback at the AR F2F
<mmocny> For those remote, here's a visual: https://lh3.googleusercontent.com/fzhkTeAssxTuWukPrdHTb-MKGAYTWb6VFtx2WkgzsIoDasBJOj1LCOZxsT5jFQOxWYMpPiiYtrxncKWasD7ck1HkYHGvJ3Ipa-WgZBXhcVMUa2tRup1C-X7z28-ZMv6rXx-QBB7NK5w=w1011-h758-no
artem: occulus definitely
interested in implementing this in VR (not AR)
... for the text presentation in VR, the cylinder layer is much
better than quad
it's much more comfortable to have something curved in front of you rather than flat
scribe: [11:58] <ddorwin>
q+ to note that there are other types of "input" to the page.
For example, intersection observer. Thus, we either need to
disallow such APIs when a frame is used this way or ensure we
are providing accurate data (and considering all the eye
tracking, etc. concerns). Also, what does it mean for a pixel
to be visible in a
... I hope we are talking about adding layers to webXR
... why limit to just one
... cylinder layer, cube map, etc
... it's more comfortable to have cylinder than flat UI
nell: not every platform supports
quad layers. To have compatibility across the web, the burden
would fall on the UA to do this
... how do we define this feature in a scoped way to be
implementable in the time frames we are looking at
... what is the smallest viable feature set that enables this
stuff?
... when we start looking at cylinder layers, etc, it starts to
blow up in time frame to implement on all UAs
... just a quad layer seems like it might be reasonable but we
still have to investigate even that
... we are looking for the smallest viable feature set to start
with
<Zakim> alexturn, you wanted to ask about <100% immersive coverage for screen-space control panel on the side, per Blair_Moz's feedback at the AR F2F
alexturner: for the screen space
stuff, Blair mentioned the way some apps use screen space today
to have control panels with like a shifted immersive view with
a fully opaque screen-space view
... does the immersive view have to take up the full view or
can we have UI on part of it
nell: we haven't really thought
about the implications on camera feed etc. in the AR context
specifically - like not preserving the aspect ratio
... I suspect (completely unconsidered thought, straight off
the dome) - question of opacity or not - could build a screen
space overlay that blocked the camera feed or one that
compressed the feed
... nothing in our designs today that would enable compressing
to a subsection of the screen
... if I recall from last convo with Blair, he said as long as
we can do hit-testing from any point on the screen that we
don't need to be able to compress into a subsection
blair: Yeah, the only weird thing
is sometimes people would create simple AR apps where people
would create AR apps with the content in the center of the view
frustum - like auto-add some furniture
... if half the screen is covered, then the center of the view
is kind of half-clipped
... the 3d math would do things in surpri9sing ways to people
but it's all dealable
<Zakim> ddorwin, you wanted to note that there are other types of "input" to the page. For example, intersection observer. Thus, we either need to disallow such APIs when a frame is used
david: with any kind of live DOM, what does it mean to be visible in terms of occlusion, etc
nell: yeah, is it an element with
a subtree, is it its own Document, is it an iframe? Still
noodling, just wanted to plant the seed to get feedback on
these topics from the group
... opening the conversation, feedback wanted!
<trevorfsmith> It's 4am and I'm going to sleep. 👋
<NellWaliczek> sleep well!!
<dom> Guido's slides on multitasking
<dom> Expose multitasking support for webXR #15
<boaz> scribe: boaz
<dom> scribenick: boaz
<dom> [slide 1: Multitasking]
<dom> [slide 2: AR Scene +3D objects]
Guido: I read the proposal #15 on multitasking and thought it was interesting and wanted to talk about use cases
<Blair_Moz> slides?
I have two use cases to present for AR Scenes and 3 opbjects; virtual vacuum robot controls and virtual advertising
[link to slides above]
the main point is the javascript used to program the a/r scene and the javascript used to drive the robot.
the a/r scene author may not have anticipated the robot, and it may be a cross origin negotiation
the second example is that there may be objects in the world that are adverts and are third party javascript programs
<dom> [slide 3: What we already have]
advances slides
guido: the way I see it today is
that there would be a main thread , and the 3rd party
javascript from the robot and the advertisement would run in
one thread with my a/r scene
... of course I can spawn workers in the background
... two issues: one the thread may have too much to do and drop
frames
... the second is, there may be xss issues
<dom> [slide 4: Solution ideas]
guido: what I want to know is
where we figure this out
... one solution i want to discuss is; is it possible to have
the xfr send objects to multiple contexts
we could call these layers and run them in different contexts and separate them from each other for the purposes of securityu
<Zakim> alexturn, you wanted to talk about how immediate composition has multidraw concerns, and how retained platforms built on top of webxr could allow multiple sites to submit geometry
alex: this is an interesting
thing going on on native platforms, too: how do we make a
platform where different authors' programs can interact.
... also we are seeing this in secondlife
<Blair_Moz> +q to summarize argon experience
alex: it feels like a lot of our
immediate mode apis are stable, but there is this really rich
exploration going on in a retained mode, where people are
walking around and their spheres are interacting and they are
running multiple authors' content
... so one way we can think about this rich time is how do we
provide concepts to authors for dealing with this
... is it DOM?
<ddorwin> s/alex;/alex:/
alex: how do we prevent people
from using native to get around sec primitives.
... so that they use the web.
mmocny: i think there is def
value in thinking about layers for multiple js contexts, but
since you refered to the preso I gave yesterday
... my proposal yesterday was that authors should present
content statically as data so that users can see it privately
and performantly
... because simply pointing your headset at the world does not
represent user intent for interacting with a third party
<Zakim> Blair_Moz, you wanted to summarize argon experience
alex: some of those people are going to aggregate content
Blair_Moz: the comment from
google points to the need for something like this.
... the only way to do this in a non google website is to run
your own app.
<dom> argon.js - A javascript framework for adding augmented reality content to web applications
<boazsender> nell: it doesn't sound like we've reached enough consensus to establish next steps.
<dom> scribenick: dom
Boaz: I'm Boaz Senders, from
Bocoup, a consulting team working with browser vendors on
WPT
... we work on developing test, maintain the CI for running the
tests
... my presence here is out of curiosity about your approach to
testing
Nell: some background on existing
discussions
... started last year in TPAC but stalled out
<ddorwin> WebXR Test API repo/explainer: https://github.com/immersive-web/webxr-test-api/blob/master/explainer.md
Nell: two sides to the
coin:
... how do UA test themselves? how do XR content creator test
their apps?
... we had been hoping to have something that would work for
both
... I had conversations with James Graham on extending Web
Driver for this
... that direction has value
... we sketched what functions might be necessary
... a number of the considerations are independent of WebXR per
se, but are linked to hardware behavior
<Zakim> jillian_munson, you wanted to provide context on the sumerian use case for browser testing, what features we need
Jill: for Sumerian specifically,
I can talk about what we would like browsers to expose
... for testing automation
... we would love to have one automation platform to drive both
browser and device events without dealing with hardware
... mocking the browser and headset behaviors
... testing targeted at the WebXR device API is one thing, but
we also want to get capture of the generated images,
performance aspects
boaz: that makes sense
... we have a proposal from one our coworker to create device
mocks
nell: for what kind of hardware?
boaz: this was contemplated for
the case of getUserMedia
... but it could be generalized to other hardware and
sensors
jill: would it make sense to do that with the generic sensor API?
boaz: that's what I had assumed
we would be doing for VR/AR, but it looks like XR is at a
higher level than these low-level sensors
... Do you have a sense of your timeline compared to generic
sensor?
Nell: the sensor that powers XR
are not exposed as generic sensors
... there may be common patterns which we could replicate
... our charter brings us to CR by end of Q2 2019
<Zakim> ddorwin, you wanted to mention WebXR Test API and our use in Blink.
ddorwin: there is a repo for the testing XR API
<NellWaliczek> link please?
ddorwin: it's specific to WebXR,
but it's not one-to-one to the API
... it could be made more generic
https://github.com/immersive-web/webxr-test-api/blob/master/explainer.md
scribe: we're using this for
testing chrome
... it's js based before wrapping it in mojo
... could be built in web driver
... we could iterate on the JS side pretty quickly before
baking stuff in Web Driver
zcorpan: can you elaborate on the testing approach in Blink? Is this mocks or actual devices?
DavidD: we would want to go
deeper into our testing, but there are issues, e.g.
security
... another option would be to expose openVR in some way
... a lot of this is implementation specific, and may be
difficult to get running in wpt.fyi
boaz: for wpt.fyi, one of the
things that we do is run the CI infrastructure that collects
the results
... on sauce, AWS, on our own servers
... we also running nightly versions of browsers
... so we could run tests on non-production chrome
zcorpan: you could also expose
things via command-line flags, where it would be disabled by
default
... e.g. to disable some security constraints
Nell: one of the challenges when
trying to build middleware on the Web for all these
devices
... there are fundamentally behavior from device to
device
... one of the things we have to do at Amazon is to validate
across browsers and across devices
... it becomes a disaster of a test matrix and thus reduce our
commitment to where the platform is going to work
... we have to encapsulate some of these different hardware
behaviors in some way to simplify this process
boaz: this may take the form of providing this encapsulation to hardware manufacturers
Nell: is there any learning from OpenXR in this space?
<boazsender> thanks everyone for having us!
<boazsender> :wave:
Issues labeled as FTF discussion requested
Default FoV for magic window canvases #272
Brandon: the question is whether
we just pick a value as a default? is this something we leave
to developers?
... or should we assume that in abssence of head tracking the
user face is .5m from the screen?
<Zakim> Blair_Moz, you wanted to related some experience
Blair_Moz: we did research around this - users tend to move their head to get closer to the screen leading to bad results
Brandon: have you used zSpace devices?
Blair: not recently
Brandon: they have a system place where leaning toward the screen produces a very natural result on the 3D content
Nell: propose to keep that as part of the continuation of the issue comments
<ada> https://github.com/immersive-web/webxr/issues/203
Expose combined frustum from both eyes #203
<Blair_Moz> brandon: I think most zspace content (that I saw) used stereo and sat above the display; in our case, we used a handheld and had the content behind the display
Nell: the discussion from yesterday could have an impact on that
<Blair_Moz> this probably had a big effect.
Nell: on multiview
rendering
... I propose we wait to get the outcome of that discussion
before picking this again
design "Lite AR Mode" that doesn't require permissions/consent equivalent to camera #29
Blair: I've posted a request for feedback on some of the ideas on the thread - got some (but not a lot of) feedback
<Zakim> Blair_Moz, you wanted to talk about simple AR mode
Blair: at a high level, the idea
is to define approaches for e.G. hit testing that can
fail
... there is a massive spectrum of possibilities which would
need user testing
... please bring feedback
Nell: maybe a good one for the CG
teleconf in 10 days
... we'll stop now given our meeting with Web Platform WG
now
Ada: we're going to floor -2 - we
will be discussing Gamepad API
... come and join us if you're interested
<daoshengmu> test
<BrandonJones> testack
<daoshengmu> Will we discuss Advanced WebXR Controller Input #392 in TPAC?
<johnpallett> Reminder: Privacy group will arrive at 3pm
<alexturn> scribenick: alexturn
cabanier: Here from Magic Leap,
startup in Florida
... device is similar to HoloLens, but has a device in your
pocket to power the device
... Created LuminOS as a custom OS here
... also wanted to let folks place things in the
environment
... immersive applications and landscape applications
... immersive is like WebXR: draw to frame buffer every
frame
... landscape applications are ilke on the screen, where they
all work together and display together
... persistent - they come back when you turn off and restart
your device
... users don't think of them as applications - just something
always there
... apps run in bounded areas called prisms
... prisms constrain the app to operate within a bounding
box
... within each prism, there is a scene graph made of
nodes
... nodes can models/images/sounds/transform nodes
... scene graph is dynamic and animatable - can be built and
then changed on fly
... apps are sandboxed and can't see each other's scene
graphs
... apps each live in their own process and communicate with
the universe
... it's like a compositor that takes data from all landscape
applications over ipc and tells apps when user looks at app or
away from app
... browser shows previews of sites and apps with little 3d
icon on top
... want to give JS devs access to this environment too
... wayfair site: web page displays on front
... has prism behind the site which JS from site can
manipulate
... for JS, all content is relative to browser window - no idea
of where user is or world is
... some people wanted custom HTML tags, but decided against
that
... instead, created custom elements and low-level JS APIs to
call into Lumin runtime
... Chris created Prismatic, which devs can include on HTML web
pages
... can then do <ml-model> or <ml-quad> and decide
where they're placed in environment
... constrain prisms on where they can be placed - if trying to
place farther away, ask user for permission
... remembers per origin if user allows use of entire
environment
... Prismatic.js calls into Lumin engine, calling Helio and
then LuminRT
<ml-model> has style and src tags and allows leaper model to appear on page and leap out
scribe: <ml-model> has
style and src tags and allows leaper model to appear on page
and leap out
... can also allow users to click on chair and extract it into
environment
cwilso: Can I grab anything to make something draggable into the environment?
chrwilliams: Can add a tag to an element on the page to make it draggable into environment
<ada> Sorry I opened the queue
<ada> cwilso: can you make sure you use a mic for questions
cabanier: Chose this model
because it's very simple
... If people run out of power with ml-model, they can go to
low-level API, but we discourage it
<ravi> yes, one page can create more than one prism
cabanier: what do people think?
Did we do the right thing here?
... Using Prismatic lets us switch out the library as we talk
further as a group
<Zakim> Blair_Moz, you wanted to comment on gltf ...
Blair_Moz: I like this sort of
thing in general - been thinking about how to enable
multitasking
... may have web pages you're not even looking at that add
critters to your environment
... would love to look further
... in the past, another issue raised is how to ensure that any
given glTF looks the same on each platform
... depends on the renderer
... don't personally think this matters that much as long as
glTF is generally a standard
<ada> +1 to tht things can look different on other platforms
Blair_Moz: curious how much back
and forth on what you allow
... can I dynamically change a texture continuously as the
prism runs
cabanier: yes, that's certainly an option
<ada> The power of declarative is allowing the platform to adopt content for the appropriate audience
cabanier: no limits, can change
at any time
... can apply color/transparency
NellWaliczek: People might
presume that since I'm so focused on an imperative API, that
I'm not in favor of declarative
... that's not true, this is a big part of where we need to go,
and is why our CG is called the Immersive Web CG
... there's a whole family of specs that will make up what
folks think of as their options
... for immersive web, folks will have many reasons to pick
among these options
... don't think you don't have my support here
cabanier: yea, that's why we pursue the custom element model
NellWaliczek: yea, for now as
people think about model formats, that may already be somewhat
well defined
... as we start to get to multitasking or cross-origin content,
now the complexity that comes in can block people
... accessibility! i could keep going
... when thinking of the problem space as what can the UA
provide that would simply not be available if it doesn't open
up an API, that's where we are most leveraged to make a sea
change on the web
... part of why i'm here in the immersive web group is to make
this kind of thing happen!
(switching to the privacy interest group's visit)
<dom> IWCG Privacy Update [PDF]
ada: Chris and Ada are the WG chairs, Nell and Brandon are the spec editors, and Trevor is the CG chair
johnpallett: I'm John, we have a
privacy-and-security repo, and I've been managing issues in
that
... could easily fill 2 hours, and we've got 30 minutes
Tara: Co-chair of privacy group, working at Google
Jason: Working at Apple
<dom> Immersive Web Privacy and Security
johnpallett: First 15m are to do
a super-fast overview of privacy for our group, both for
privacy team and those in our group who are not focused yet on
privacy
... deck is long and we won't fully go through it but you can
read the deck online
... still more detail in the repo in the explainer
... want to open things up for bidirectional discussion, and so
we'll then open this up to you
... WebXR is an API to let apps build VR and AR
experiences
... from VR where everything is virtual to AR where you add
content to the world
... API gives developers device poses for rendering and gives
real-world data for augmenting the world
<dom> [slide What is WebXR]
johnpallett: on an AR display, the UA manages the camera composition, but you get to render your own content
<dom> [slide Privacy Approach]
johnpallett: also supports
see-through displays, where you see the real world
optically
... our privacy approach is to identify XR-specific threat
vectors
... WILL NOT COVER EVERYTHING
... want to highlight some key observations our group has
had
Jason: Are you tracking threat vectors in one doc/issues?
johnpallett: Tracking as issues for now, and will move into explainer
<dom> [slide Threat vectors]
johnpallett: Threat vectors: 1) *Perception* of camera access
<dom> [Slide Why does it matter? [perception of Camera access]
johnpallett: In AR, the user may see the camera rendered and believe the site can access the camera, even if we did prevent pixel access
<dom> [slide Perception of Camera Access]
johnpallett: Even without camera
access, site could pop up a screen to appear like it can
capture picture and blackmail user
... Even on see-through devices, app could show a record
indicator and pretend like it's recording, even if device
doesn't have camera
<dom> [Slide Real-World Geometry Data]
johnpallett: Mitigation: Could
communicate to user that site doesn't have access
... to camera
<dom> [Slide 10 Why does it matter?]
2) Real-world Geometry Data
scribe: 2) Real-world Geometry Data
<scribe> ... new APIs give apps access to real-world shape of the user's room
<dom> [slide 11 Threat Vectors]
UNKNOWN_SPEAKER: good because it
lets apps lock objects at precise places in real world
... threats: apps can get PII: face analysis, embossed credit
card #
<dom> [slide 12 Threat Vectors]
UNKNOWN_SPEAKER: could
fingerprint using room geometry
... locate user even without permission
... could profile income/height/previous visited
places/bounding boxes/etc.
<dom> [Slide 13 Potential Mitigations]
UNKNOWN_SPEAKER: could access
historical data from previous sessions where there may be
ghosts of previous room planes/data that the user had cleared
away
... Mitigations: throttling/precision doesn't really mitigate
problems here, only hurting the experience
... low-res data might be visualized by the user - how do we
tell the user what's going on?
... can we ask the user somehow whether to just do low-res
data?
<dom> [Slide 14 Object Identification]
UNKNOWN_SPEAKER: opportunities for AR subsystem to filter the scope of data - perhaps just new data for new session
<dom> [Slide 15 Why does it matter?]
UNKNOWN_SPEAKER: 3) Object Identification
<dom> [Slide 16 Threat Vectors (Object Identification)]
UNKNOWN_SPEAKER: in AR, good to
find known objects to annotate them
... problem: user can't easily understand what's being scanned
for
... could find location through storefronts/qr code/street
sign/configuration
<dom> [slide 17 Object Identification POssible mitigations]
<dom> [slide 18 Permissions]
mitigations: for keeping users safe (prevent blocking stop sign), could perhaps limit access to precise position?
<dom> [back to slide 16]
Jason: In the first threat vector, sounded like they couldn't get camera, but now they can?
johnpallett: Site could perhaps
ask user agent "tell me when you see cup" and that could give
data to tapp
... or could build an api that uses camera to find poster,
still without actual camera access
Jason: So should we presume apps don't have access to camera?
johnpallett: Don't have a conclusive answer to whether apps will get camera access
NellWaliczek: Currently shaped to not give camera data
Thanks to John for doing great work on due diligence here
scribe: Thanks to John for doing great work on due diligence here
<dom> [slide 18 Permissions]
scribe: some of this is actually further out than it might seem
johnpallett: 4) Permissions
<dom> [slide 19 Permissions]
johnpallett: this kind of wraps
up a lot of the topics we covered
... worried about permission fatigue
... can i have camera? world data? water bottles? say yes to
everything
... users may not understand all these implications
... other question is around scope/duration of
permissions
... may be easy to understand wanting to enter an AR
experience, but may not expect that to be permanent
... exploring idea of an AR mode
... will be immersive not inline so that it's explicitly
permissioned
... have explicit session scope to enter/exit with user
intent
<dom> [I'll note that this "session mode" is similar to what is done with screen sharing in webrtc]
johnpallett: also gives UA more ability to show custom UI
NellWaliczek: Should explain immersive in context of handheld device
johnpallett: Like full-screen: a
mode you enter and exit
... on headset display, would render the scene without
rendering anything else
<dom> [slide 20 Pose and Frame of Reference Data]
NellWaliczek: Dom mentioned it's similar to screen-sharing in WebRTC
johnpallett: Powering through in remaining 11 minutes
<dom> [slide 21 Access to Pose and Frame of Reference Data]
johnpallett: In order to render
3D geometry, device position and orientation are necessary to
provide
... Knowing how high off of floor and space bounds within which
user can move
... Local frame of reference - doesn't by itself give
real-world location
... could be on phone/laptop/headset
<dom> [slide 22 Threat Vectors]
johnpallett: trying not to box us in based on form factor
NellWaliczek: 2D browsers as well
johnpallett: Headset displays
also can show 2D browsers in 3D environment
... that web page could then use the WebXR API
... to show a 3D element within a larger 2D space
... other threats:
... Gaze tracking to know what the user is looking at
... Should pages be able to know what you look at outside the
page?
... Device orientation will allow you to infer what keystrokes
are being entered
... Could read PINs on phones with orientation at 20hz
... could affect gaze on virtual keyboards
<dom> [slide 23 WebXR Modes, Implied Threats]
johnpallett: location profiling: can determine location within city with 400m of trajectory trace
<dom> [Slide 24 Potential Mitigations]
johnpallett: some frames of
reference restrict what data is available
... input sniffing is tough: a lot of mitigations look similar
to generic sensor data
... allow data only to focused and visible page
<dom> [slide 25 … more generally...]
<dom> [slide 26 Potential Mitigations]
johnpallett: if sitting in chair,
may be able to limit bounds of pose data
... other mitigations for gaze tracking such as blurring
things
<dom> [slide 29 Join the conversation!]
johnpallett: please do join the conversation if you're not part of this yet!
<Zakim> NellWaliczek, you wanted to define gaze tracking
NellWaliczek: gaze tracking and
head pose tracking may not be the same
... may be different threats with eye vs. head direction
johnpallett: Privacy group: Sorry for firehose! Do you have any Qs?
Tara: Thanks! Blown away by sheer amount of threats but you've done so much work on mitigations!
<dom> Self-Review Questionnaire: Security and Privacy
Tara: Looking at similar things with sensor API, and would love to know how you got to this exhaustive list.
johnpallett: Haven't yet found
big book of XR threats
... some existing research into these areas
... may be areas we need to research ourselves, such as user
prompts
Tara: Snickering because we have
the same item on our agenda
... you may be one step ahead of our own agenda
johnpallett: Ideas on how we can best engage with you?
Tara: We have our own github with
privacy questionnaire - a lot of things take place as
conversations
... do teleconferences once a month, can review materials ahead
of time and give you an hour to bang on issues
... given amount of issues here, would make sense to have a
longer conversation
ada: Almost out of time...
weiler: Where are you on hard mitigations such as not putting data into the API or not making it available?
johnpallett: In terms of our process, we're doing it in three steps: 1) threat vectors 2) potential mitigation 3) specific language
NellWaliczek: Could take a stance
and decide not to give data, but you can, in essence, get
everything from camera
... could generally be safer to encourage use of non-camera
streams for most data streams
johnpallett: One idea of a hard mitigation is inline vs immersive for frames of reference
NellWaliczek: Not sure what you're referring to here?
johnpallett: There are multiple
frames of reference a view can use that result in different
levels of data access
... some only available when in immersive mode
NellWaliczek: Still have valid
reasons for override option - for diorama mode
... so may not be as simple to secure inline mode
<Zakim> NellWaliczek, you wanted to discuss future collboration
johnpallett: Would like to know how we can best help you
Tara: Don't have a super strict method to impose - don't worry too much about that
weiler: Like the holistic threats that emerge when you look at the whole picture
Jason: Holisitic is best to start for those of us new to XR
NellWaliczek: To help get context, may be best to do quick demos now before future calls
<ravi> Thanks John Pallett
<jillian_munson> scribenick: jillian_munson
cwilso: declarative models, had
an offline conversation with ML team
... 1. Momentum behind gltf, glb being a baseline interchange
format
... Feel that this is not an industry consensus, wanted to ask
the room for thoughts
Rik: Yes, this is most requested format from developers.
<Artem> +1
<max> +1
<NellWaliczek> +1
<alexturn> +1
<ravi> +1
<cwilso> +1
<alexismenard> +1
<lgombos> +1
<dom> +1
<cabanier> +1
<Blair_Moz> +1
<ada> +1
<chrwilliams> +1
<arno_> +1
<DanMoore> +1
<Blair_Moz> note: we'll have to deal with versions
<alexturn> +2.0
<ravi> version 2.0
<ada> 💘
nell: distinction, only talking about gltf 2.0
<Blair_Moz> As long as it's defined, "any version" will do
cwilso: Yes, also in favor of declarative solutions. Focusing on basics of WebXR, this is not necessarily where we are going to land, open to declarative interface
<dom> The Extensible Web Manifesto
cwilso: Important to figure out
core functionality of API to build on top of
... Comment on Magic Leap implementation: assumes that there is
an initial pose, orientation for rendering
... May need to think about doing this in context without that
position data
<Blair_Moz> Nell, sure. The point is that as long as people using it know "2.0" or "2.1+" or whatever, and models created with 1.1 are identifiable as such, it's "fine"
cwilso: wanted us to start
thinking about these use cases for models, where we should be
incubating them. If not a proposal, at least a proof of concept
to start the conversation
... how we should incubate this
<alexturn> I think glTF 1.0 and 2.0 have the same MIME type, so it could be dangerous if some browsers start randomly supporting glTF 1.0 and we all end up compelled to do so, since it's not really easy to secure
cwilso: but in the short term, focusing on the device API
ada: Really excited about what the Magic Leap team has developed so far
<johnpallett> +q to say that we should also document existing declarative 3D model approaches in addition to use cases
max: Agree with Nell, this is a
huge topic to unpack
... Our team has also been exploring the declarative space,
thinking more broadly on how to deconstruct pages into elements
in world space
... interesting features - progressive enhancements for pages
that can transition between experiences, experiences with 2D
elements embedded
... very interested in this, great start to the
conversation
<dom> LensVR The VR/AR web browser
max: Would love to set up a presentation with folks not here from team to demo for this group
<Zakim> NellWaliczek, you wanted to discuss apple's involvement
NellWaliczek: lots of designs
around things that try to be declarative systems, flavors of
declarative design
... some confusion in the ecosystem today, would love if we
could proactively reach out to our developers
<ravi> Max is this the one you were talking about: http://www.joshcarpenter.ca/declarative-3d/
<Zakim> johnpallett, you wanted to say that we should also document existing declarative 3D model approaches in addition to use cases
NellWaliczek: run the risk of fragmentation, we should set up a conversation with Apple to discuss what's the same, what's different, get consensus
johnpallett: Would be good to
capture some use cases, different declarative approaches that
have been implemented
... beyond just use cases today
<johnpallett> johnpallett: wants us to list what's been tried before, capabilities, pros and cons so that we can learn from past experiments
alexturn: Example, threejs,
a-frame also declarative-type models even if they are driven
entirely in javascript space
... should consider these use cases and what they have done as
well
<johnpallett> +1 thanks Alex
mmochny: Assuming declarative,
browser understands the content, etc.
... Is the model handled by the page? Pose data?
rik: No, once it gets generated
the model belongs to the system not the page
... can close the page, model persists and belong to the user
space
mmocny: With respect to pose data, the page does not need to know how that is done?
rik: correct
<Zakim> NellWaliczek, you wanted to talk about khronos
rik: Specifically used custom elements, if community comes up with something else we are willing to switch out underlying implementation
NellWaliczek: one thing to be
aware of, gltf 2.0 is being driven by khronos
... if there are things that are inadequate, changes we want to
request, logistical complexities
... there are folks in this group that also members of khronos
to facilitate
... have a model of how we communicate across standards body,
need to be sensitive
alexturn: Microsoft is a member
of khronos, requested extensions to gltf
... could we request an extension if we are not part of the
group?
NellWaliczek: My guess would be no
<dom> glTF Extension Registry
NellWaliczek: Similar conversation yesterday around WebGL and support functionality we need
<dom> "To create a new extension, use the extension template and open a pull request into this repo. Make sure to add the extension to the glTF Extension Registry (top of this file)."
NellWaliczek: we can use that to
figure out how those conversations should happen
... Brandon is in both groups, will follow up with him
<dom> "Extensions start as a vendor extension, then can become a multi-vendor extensions if there are multiple implementations, and can become a ratified Khronos extension (the multi-vendor extension is an optional step)."
ada: Any last comments for this topic?
<max> thanks to Dom for doing all the random IRC wrangling
<alexturn> +30
cwilso: Wanted to get a sense of whether or not we could change the meeting time to be on the hour and not on the half hour
ada: If you have the existing meeting invite, please delete
<DanMoore> +!
<Blair_Moz> on the hour is fine for me
<DanMoore> +1
<gen> +1
cwilso: Wanted to get a sense of whether or not this was a shared problem
<alexturn> +60
<ravi> +1
<NellWaliczek> +1
<Blair_Moz> +1
<chrwilliams> +1
<johnpallett> +1
<max> +1
cwilso: +1 for on the hour time zone
<alexismenard> +1
<Artem> +0
<alexturn> +90
<Blair_Moz> We should do a doodle poll on +/- 30 minutes
cwilso: expect a doodle to change the time for the meeting
<gen> earlier please ...
<cwilso> ACTION: chris to send out a doodle poll to change time.
ada: we will do the poll, sorry
if you are unable to make it
... tuesday's meeting is a working group meeting
... agenda has gone out
<dom> [regrets for Tuesday from me as well]
NellWaliczek: Won't be there, on vacation
ada: if you want to add
descriptions for agenda topics, please open a pull
request
... has all call details, tuesday at 6:30 UK time
dom: Also, there is daylight savings...
ada: Meeting is pinned to west coast time
cwilso: also, there is a F2F in january from 1/29-1/30
ada: It will be in Samsung
America
... and now you are all free
