– DRAFT –
WebVR CG
09 November 2017
bjones: are remote participants fine with the connection?
anssik: what is the status re specs and implementation, and next steps for WebVR CG?
bjones: welcome everyone to this ad-hoc WebVR CG meeting
… WebVR API allows interaction with VR and AR hardware, we have mainly an explainer document currently
[bjones giving spec overview]
... we are at the explainer stage now, no full spec yet
... we're iteration on the explainer still quite a bit, the concept has been these for 3-4 years, more actively working on WebVR for ~2 years
nell: kickoff for WebVR standardization was the W3C WebVR workshop last Oct
bjones: clarification, there are capabilities on multiple browsers now called WebVR, but it is the non-webby version
… on that (1.1) version we received TAG feedback on
… the latest spec is WebVR API "v2.0"
nell: common questions, who's working on what
… there's now WebVR 1.1 implementation on Chrome as an Origin Trial
… and Edge implementation for Mixed Reality headsets
… also in Firefox for Oculus Rift and HTC Vive
… Oculus has a Chromium-based browser, and Samsung has a Samsung Internet browser
anssik: how about Servo?
bjones: not familiar with Servo, know Mozilla has used Servo as a testbed
anssik: what are the next steps for the WebVR CG?
bjones: as far as feature wise, there's very little question next year would like to transition to WG and formalize the spec
… another thing to clarify, there was a proposal for a Working Group charter we pushed back on that, not because we object the idea of WG, but though we were not ready
nell: worth calling out, there are unresolved issues still, the API shape is still evolving and undefined
… for example, we don't have our input story
… site to site navigation, rAF relationship are still unclear
bjones: for MVP we do not explicitly add AR features, but allows AR extensions
anssik: core WebVR would move to WG, and AR features would be evolved in CG? correct?
nell: right
<dom> RAF Loop Syntax discussion
<dom> TAG Feedback: isolation and the document
<dom> Test API Design and WPT
[nell gives an agenda workthrough]
nell: out of scope today are input, gamepad
nell: related, Msft joined OpenXR in Khronos last week
<dom> RAF Loop Syntax discussion
bjones: background on the topic is that when dealing with VR devices within WebVR spec, they may run at a different frame rate than the PC connected to
… we want to cater for that, as we defined the spec it seemed difficult for various reasons to change the behaviour of rAF
… went for a WebVR variant of rAF that'd run at the VR device's frame rate and not the PC's frame rate
… got feedback using the same name might be confusing, so renamed the VR rAF variant
<dom> Offscreen canvas spec
bjones: also Offscreen Canvas is looking at creating a rAF variant called commit()
slightlyoff: explains the rationale for Offscreen Canvas rAF variant
nell: concerns that Offscreen Canvas, making sure it is setup to be as compatible as possible for exporting existing code bases
slightlyoff: had discussions re Offscreen Canvas, game developers' feedback would prefer an API that supports async/await style
… can we make rAF return promise?
bjones: no concerns re object allocation, having rAF return a promise is backwards compatibility issue, cannot pass to cAF
… would need an alternative version that is cancellable
… bigger point, you can wrap it into a promise in JS
bjones: not clear promise-returning rAF would make developer lives easier
… this is a VR-specific issue (that a promise-returning rAF does not lead to better developer ergonomics)
… that was the background
… the issue #141 discusses the proposal for new rAF loop syntax, considers TAG feedback received
<dom> [I think you would want to make AnimationFrameProvider an interface mixin https://heycam.github.io/webidl/#using-mixins-and-partials ]
bjones: the proposal for rAF as is, make separate interface that is implemented by window interface, want to extend the callback to promise timestamp and framedata plugged in there
[bjones walking through the proposed IDL on big screen]
anssik: does this consider TAG feedback?
bjones: yes
travis: I like this rAF model is extensible, not sure about Transferrable part, perhaps we do not need it
… generally I think it is great, then again, there's TAG feedback re isolation and how to make this fast
nell: we have a separate topic for that
dom: not sure whether you can override the second parameter as proposed
… WebIDL may not have semantics for doing that
bjones: not familiar with these WebIDL details, but can say with confidence JS don't care about this detail (parameter override)
travis: proposed fix option a) do nothing b) define a derived callback
travis: workaround is to just define a more precise behavior of the callback
nell: the bigger question, is this something to be addressed by the WebVR group or by the broader community, propose we find a way that does not block us
… who's the owner of this discussion so that WebVR group does not own this discussion
travis: rAF is defiend in the HTML spec, so they own the broader issue
nell: always returning zero for timestamp should be fine
anssik: what is the reference for timestamp?
<dom> requestAnimationFrame spec
bjones: not sure if it is even well specified for window.rAF
<dom> "When the user agent is to run the animation frame callbacks"
anssik: related, generic sensors have rAF sync as a v2 feature
<tidoust> requestAnimationFrame in W3C HTML
travis: what are the core use cases Offscreen Canvas spec editors are after
slightlyoff: high perf low power video hardware overlay
… my question to them was why not expose the same to main thread, it appears they want to avoid all compositing
… they got 10ms roundtrip improvement per frame
travis: are they in we want this but have not yet specced?
slightlyoff: they are not yet fully there yet
nell: considering we want to advance to more stable spec in near future, having unstable dependencies would hold us off
proposed RESOLUTION: submit pull request to change the signature in callback, move the rAF design proposal away from WebVR into HTML spec(s)
Resolved: submit pull request to WebVR explainer to change the signature in callback, move the rAF design proposal away from WebVR into HTML spec(s)
TAG Feedback: isolation and the document
bjones: we started with workers support, we do not talk about this in the explainer, but we have stripped away all window and DOM deps
… we expect input for this system to come from device specific controllers, to be able build experiences that run purely in a worker
… the issues that we run into are more about how other APIs interact with workers e.g. input e.g. mouse/keyboard is functional but slow
… latency of keyboard or mouse cursor not as important
… video is trickier, it is a large use case for VR, and there's no native way to render video in a worker
… there's a workaround but it is tedious and slow
… audio and video sync would be impossible in a worker context
travis: WebGL and WebVR running in worker on issues, input unclear, video no solution in sight
nell: is this issue addressed by iframe isolation instead?
bjones: we're dealing with three different variants of devices: 1/ mobile device in a harness, 2/ standalone HMD device, 3/ external MHD that rely on PC
… from perf isolation standpoint, there are not many cases for 3/ where you view DOM at the same time with VR experience
… as you go to the VR mode in 1/ and 2/, we can turn off general compositor
travis: the canvas you start writing to in your rAF loop, it becomes exclusive mode?
bjones: in the new version of the API, we provide a new frame buffer for the VR device itself
bjones: we can fudge and note, PC requirements for this are magnitudes larger than for mobile
bjones: ultimately we'd like to have a better solution for PC-driven environment
slightlyoff: if you want to use iframes, they can be different process
… iframes are used for many use cases, focus on iframe could be used as a hint to move into a high performance mode
<slightlyoff> specifically we can have a new attribute on iframes to say "this is high performance content"
travis: re moving to iframes, does it mean nothing changes in the spec?
slightlyoff: that'd be my suggestion
… I prefer deprioritizing content that is not displayed
slightlyoff: give the browser ability to add VR button, if the page advertises it has the capability
bjones: we do not want cross-origin iframes to use VR features
[insert Pepsi joke here]
bjones: the idea whether a particular document is in a VR is well defined
… we have clear boundaries when throttling could begin, [the button] sounds very UI specific optimization
nell: if developers are worries about performance, they will think about these things; many perf challenges are not VR specific, developers use iframes for perf isolation already in general
slightlyoff: if no-one tells us this content runs in high perf mode, browser could throttle times or turn off specific APIs
nell: VR focused state could toggle those behaviours
slightlyoff: wasm based games would prefer from such a high perf hint
… possible ways to do it e.g. feature policy, "allow=highperf"
travis: I think we have addressed a bunch of TAG concerns
slightlyoff: thanks for paying attention to TAG concerns raised, helping us understand your API better
bjones: I would far rather have these assumption breaking feedback that no feedback at all, nice to work with TAG and others familiar with web platform
nell: thank for all the feedback that makes us be part of the cohesive whole
[30 minutes left]
nell: when we first talked about interop testing, wanted to validate browser implementations
… now we've noticed the price for VR hardware going down, cannot buy every possible hardware out there
… so as we're looking at animation testing, e.g. A-Frame, look at validating content experiences
… will sit down with Web Driver people from Mozilla, to see what extensions are needed for WebVR simulation
… will provide a draft up for review
… re mocking, implication on a system level e.g. on Windows level how input is routed changes
… other thing: modeling user input, putting your headset on
mfoltz: Presentation API needs similar domain-specific extensions for testing
nell: would like to get consistency with other efforts, so good to synchronize efforts where sensible
dom: is this about instantiation only, or is testing of user interaction also in scope?
nell: both are in scope
dom: generic sensors have a need for a bunch of interactive test too
nell: things that are already on roadmap: 1/ clarify display types: pass-through, see-through, 2/ input story incl. VR controllers, 3/ site-to-site navigation
<dom> https://github.com/w3c/webvr/issues/254
nell: further out are: 4/ AR environment understanding: raycasting against world, creating anchors etc., 5/ media layers e.g. 360 video
… 6/ clarify handheld vs. face-worn
<dom> https://github.com/w3c/webvr/issues/145
nell: 7/ permissions: how to create permissions UX that makes sense to the user, e.g. how to provide permissions that are needed prior to entering VR experience
dom: there's oncoming workshop for VR content authoring in December
<cwilso> https://www.w3.org/2017/09/webvr-authoring-workshop/
<dom> W3C Workshop on WebVR Authoring: Opportunities and Challenges
Succeeded: s/WebVR/WebVR standardization
Succeeded: s/that rAF/the proposal for rAF
Succeeded: s/thought/through/
Succeeded: s/pull request/pull request to WebVR explainer/
Succeeded: s/have/have a need for