Position statement

Background:

I am a Scientist Innovator at the Netherlands Organisation for Applied Scientific Research (TNO). Currently at TNO we are looking into applied research in multimedia and shared experiences in VR. Particular this means we want to bring people together in VR, looking as real as possible. One focus is to look into what is possible to do with common of the shelf hardware. Overall we believe that the web can be an enabler of wide access to VR content, as it makes it easy for the end user to join VR experiences, without the need to install special software. Personally, I am involved in building prototypes, based on web technologies, that allows people to interact and communicate in interactive experiences.

Topic: Displaying and interacting with multi-media content and People in VR

With VR hitting the consumer market this year it is evermore becoming a hot topic in research and industry. Many new devices entering the market with many new players who all what to get their cut of the money predicted to be in VR. However, the challenges are also still big, as VR opposes a complete paradigm shift in in terms of consuming multi-media content. This is because the way VR is perceived through head mounted displays is very different from a traditional 2D screen. Both in terms of visuals and interaction models. Thus any existing content and experience has to be redesigned to match the requirements of VR. Furthermore the technology also has to match and catch up with this new requirements. Saying this content and interactive content is still beginning to emerge. The biggest gap in this respect we see in social and shared aspects of VR. In this respect the topics we are most interested in are interactive content and shared experiences in VR.

Interactivity – VR offers a complete new dimension of immersion in content. Thus there are many challenges to move from static 2d content towards VR:

    What is the necessary in terms of meta information and language to place, maintain and map (special visual and special audio) multi-media objects into a VR space?

    How can we keep all the media objects synchronized?

    How can we describe interaction possibilities with all or individual objects according to the limitations of your end devices?

Shared experiences – One key aspect in VR, as we see it, are social interactions. It has do be possible to interact with other people inside and outside of the VR space rather than making VR a private and detached experience. Just because you get into another world with some glasses on does not mean that you are alone. Furthermore avatar based approaches are to limited for real interactions in many use cases, like video conferencing, presentations, watching 360 degree videos together, and many more. We want to bring people together in VR, looking as real as possible. This adds another dimension to the technical challenges mentioned above, as you don’t only need to synchronize and offer interaction possibility with objects but also with live streams of other participants (or people in the same room).

    What does WebRTC offers for VR?
    What has to be added or changed for WebRTC to work well in VR?