W3C Workshop: Augmented Reality on the Web 15 - 16 June 2010, Barcelona

• About W3C
• Call for Participation
• Papers
• Report
• Agenda

Workshop Report

Introduction

This W3C Workshop was held to discuss whether and how the exciting opportunities offered by Augmented Reality can benefit from Web technologies. Augmented Reality on the Web was hosted in Barcelona by Escola Tècnica Superior d'Enginyeria de Telecommunicació de Barcelona (ETSETB) at Universitat Politècnica de Catalunya (UPC). It attracted over 40 attendees and 22 papers. The participants represented a broad range of businesses including telecom operators, device manufacturers, AR content and platform developers, AR users from the advertising world, academics and standards bodies.

The agenda provided time for everyone present to participate in the presentations and discussions. The primary outcome of the workshop is the strong desire to form a new W3C working group to:

1. create a standard for representing multi-dimensional Point of Interest data;
2. survey and report on how AR standards can take into account and benefit from work ongoing in other areas such as privacy, linked data, geolocation, device APIs and more.

The remainder of this report summarizes the presentations and discussions. More detailed minutes are available from day 1 and day 2.

Jonghong Jeon's photographs of the event, several of which are used here, are available in small and large sizes.

What Needs to Happen for AR to Become an integral element of the Web?

This was the central question of the workshop and the focus of the first session. Layar (AR service provider), Sony Ericsson (mobile device manufacturer) and Canon (image specialists) set out their use cases for Web standards work, reinforcing points made in the opening paper & presentation by ETRI. A highlight of that paper was when authors Jonghong Jeon, Sunghan Kim and Seungyun Lee identified 12 issues to be addressed before AR content providers and AR User Agents can truly be interoperable. Since these were referred to by several speakers throughout the workshop it's appropriate to list them here:

1. AR content markup & format
2. AR content transport/interaction method
3. Representing 3D interactive AR content
4. Event Scripting
5. Local caching method
6. Additional functionality of AR user agent
7. Device capability access APIs
8. AR data mashup methods
9. AR data format - POI (images, text, 3D models, URIs)
10. AR data service API
11. Open marker database
12. Security & privacy

(See slide 14 of Johnhong's presentation).

Considering all the presentations in the first session, it was clear that consensus was already beginning to coalesce around the need for a standard method of representing Point of Interest data as a minimum, although several other areas are of key interest. Notable among these is the need for rich device APIs, exposing the functionality of devices and their sensors to Augmented Reality services and applications. This is particularly true for mobile devices.

What Role for Standards?

The workshop focused on the general topic of standards for AR and the Web throughout so distinctions between the theme of each session is necessarily blurred. However, the second session looked at the kind of standards and data interactions that might help the advancement of AR. Participants presented a range of ideas on how AR can be seen in a broader context: social networking, the uploading of relevant data, and context-based filtering of the mass of data available and potentially relevant to a given user at a given location at a given time.

These themes are familiar from the Web as we know it today but bringing that paradigm to Augmented Reality is, as yet, largely unrealized. Some sort of extension or new application of existing standards is required.

Wolfgang Damm from Mobilizy (makers of Wikitude) presented a comparison of two efforts to create a standard to describe Points of Interest. One, ARML, was produced largely by his own company and is used in the Wikitude browser. Another, KARML, was developed by a team at Georgia Tech lead by Blair McIntyre as part of the KHARMA environment. The latter uses a lot of familiar Web technologies to create an open platform to which users can contribute in much the same way as the LibreGeoSocial Open Source project presented at the workshop.

An e-mail thread directed at the workshop that includes contributions from some of the people close to the KHARMA project and others can be seen on the Open AR Googlegroup.

In his paper, Jens de Smit pointed out that much of the processing behind current AR applications can be and is done using standard Web technologies. Further standards are required, however, such as full access to the camera, accelerometer and compass within mobile devices, if the Web technologies like HTML 5 <video> element, JavaScript are to support AR fully. Current W3C work on, for example, the camera API, is clearly important in this regard.

AR in a wider Context

The session entitled "Data, Reality and Things" looked at the subject of Augmented Reality from different perspectives. In their paper, BDigital posited AR as an enabler for the Internet of Things. Connected Environments work in this space too through their product Pachube (pronounced "patch bay"), which can use AR to visualize a variety of sensor data. Chris Burman presented a list of requirements for AR that includes some items not discussed in earlier sessions:

• Markers or 'hooks' for orientating content. Do we really need to stick QR codes on everything?
• Image recognition/processing. Generic surfaces like desktops are really hard to process).
• Object-scale (rather than "city scale or global scale") geo-location.
• Efficient caching of 3D interest (mobile Web 3D is still a bit rubbish but it's coming!)

If AR is to develop further as a key tool for city planning then sensor data, object data and more will perhaps need a common infrastructure but different users may need different levels of access to the data.

This segued into the presentation from DERI about the potential for linked data in AR. Vinny Reynolds et al observed in their paper that some aspects of the proprietary systems currently used in AR browsers won't scale. How can new data be discovered and integrated into a particular augmented view? Available contextual information is generally under-utilized. Linked Data has a great deal to offer in this space. Current estimates put the number of data items available at 20 billion and it's growing rapidly with governments publishing more data (such as through data.gov and data.gov.uk) and the adoption of RDFa by Facebook and other major players. Much of this data is already geolocated through GeoNames, LinkedGeoData and DBPedia.

There is a tendency to think of AR information superimposed in the user's visual field. Jacques Lemordant of INRIA presented a very different take on AR - Augmented Reality Audio. The sound capabilities of the iPhone and similar products makes this all the more feasible for applications including audio walking tours, geolocalized games, navigation systems and, for the pure aesthetic pleasure of it, soundscapes. Using a dialect of XML called A2ML, that takes its inspiration from SVG, it's possible to cue specific audio files or mix several files to generate complex sounds in response to events. Work with a very similar theme is also being carried out in Barcelona by Nathanial Finney and Jordi Janner of Universitat Pompeu Fabra.

So far all the discussion about AR on the Web has made the implicit assumption that the HTTP protocol will be used to transfer data from server to client. But is this appropriate? Thomas Wrobel, for one, thinks not, preferring Google's Wave protocol. For all its strengths, the mood in the room, however, is that content publishers at least will be much more comfortable with what they know (i.e. HTTP).

Augmented Reality Gets Real

The final paper-based session centered on commercial aspects of AR. It began with a reminder of the potential social impact of widespread AR from Dan Romescu, and ended with the HCI Computer Group from the University of Oviedo presenting the SHUBIA Project that applies AR to an area where there is substantial potential for development - providing supplementary information for disabled people.

Other contributions in this session looked at AR prototypes and industrial solutions other than those commonly found in an 'AR Browser' such as those in use and under development at IBM together with medical and building industry applications. It's clear that the collection of technologies loosely defined as being part of Augmented Reality do have very real commercial applications with significant potential for future profitable growth. The possible business models for AR on mobile were examined by Manel Medina et al.'s paper].

The GeoBuyMe project, which worked with QPORAMA, provides a solid use case. A variety of marketing and social-media tactics were used within an AR context, essentially alerting users when they pass within a short distance of a store that sells items they have added to their wish list. Printed promotional material including both coupons and point of sale displays carry QR codes (markers) and, depending on things like how often a user has recommended the GoBuyMe service to others, they receive smaller or larger discounts. Additional commercially-attractive use case for AR and the requirements of industrial users were discussed by Total Immersion, ARPA Solutions and the Young Generation, who were all represented on the final panel.

Demos

At the end of the first day those advertising and marketing-based companies offered a series of striking demos of Augmented Reality in use. Other demos were presented by Claudio Venezia of Telecom Italia (paper ) and Timo Engelke of Fraunhofer who showed an iPhone application that used various Web technologies including X3D.

Before the end of the demo session, the workshop members saw a video demo of the Acrossair AR Browser. Close attention was paid to this video - Acrossair sponsored the dinner!

Discussions

In order to generate as much discussion including as many people as possible, the workshop split into three groups.

The three groups discussed the AR landscape in the context of the Web and standards. They developed what they felt were the most pressing issues or matters of highest risk.

Recurring themes include:

• how to publish Point of Interest data once for representation across many AR services, a common AR data "tagging" standard;
• the importance of 3D (including X3D, the Web 3D Consortium and others)
• the opportunities for using and reusing Linked Data;
• how to make it easier to deliver the best user experience;
• substantial and repeated concerns over privacy;
• Jonghong's 12 points;
• what is a point of interest? An object? A location?
• interaction can come from geo-social content with crowd sourcing to increase confidence levels;
• inputs from other sensors, haptics, temperature can be important;
• existing standards need to work together with new standards;
• what is the origin/provenance of the data?
• temporal dimension as important to understanding of context as geolocation;
• AR is currently defined by what we have rather than what we want to do.

Call to action

During the event, and especially during the discussion during the final two sessions of the workshop, it became clear that the need for a standard for publishing Point of Interest data was pressing. After considering the various options, it was agreed that work on such a standard should, if possible, begin without delay. Jonghong Jeon pointed out that this is also of interest to other relevant organizations such as the Open Mobile Alliance (OMA).

The minutes from this discussion show clear support for a full Working Group with two principal aims:

1. To develop a standard for representing Point of Interest (POI) data.
2. To contribute to the industry a Working Group Note on how AR standards can take into account and benefit from work ongoing in other areas such as privacy, linked data, geolocation, device APIs and more.

To facilitate the evolution and future working of the group, a public mailing list has been established (with archive) to which you are encouraged to subscribe!

Finally, the participants thanked our host, professor Manel Medina of UPC, the sponsors of the conference dinner, Acrossair and recognized the important contribution made by the OMWeb project which is funded by the European Union through the Seventh Framework Programme (FP7/2007-2013) under grant agreement n° 248687 - Open Media Web (OMWeb).

Phil Archer $Date: 2010/07/22 09:01:06 $ by $Author: phila $ with additional input gratefully received from Christine Perey.