Utilisation of Wireless Application Protocol to Implement Mobile Augmented Reality Based Services

Tuukka Turunen, Tino Pyssysalo and Teemu Lankila

Computer Engineering Laboratory, Department of Electrical Engineering

University of Oulu

E-mail: tuukka.turunen, tino.pyssysalo, teemu.lankila@oulu.fi

Abstract

In this position paper we describe some of the key issues in using the wireless application protocol for mobile augmented reality applications. We underline the importance of both position and orientation information in these type of applications and suggest a format for their representation. We also describe our ideas for some service scenarios that utilise mobile augmented reality and future mobile terminals.

1. Introduction

Mobile Augmented Reality (AR) makes it possible to create new kind of services and applications. Some example applications of mobile AR include personal navigation, guidance systems, teleoperation, security, entertainment, e-commerce and personal services [1, 3, 4]. We define AR to mean any case where computer generated objets (text, pictures, sound etc.) are added to the user’s perception of the real world. By making AR work in mobile environments we face new challenges in positioning, registration, system performance and energy consumption. Some experimental mobile AR systems have been built, but most current systems work only in a restricted area and do not have sufficient means for communication. They usually use commercially available or self-built wearable computers as their platform and their application area is often limited to some specific tasks.

The evolution of cellular networks and related technologies has recently been very fast. The transfer rates and low latencies for packet data required by many AR applications are feasible using the upcoming 3rd generation cellular networks. The processing power of yesterday’s supercomputers is now found in children’s toys and is soon integrated into mobile terminals. This together with evolving transfer speeds, miniaturised displays and accurate positioning enables the building of a commercial mobile terminal with sufficient capabilities for mobile AR in the near future. To ensure the interoperability of these systems, common standards for communication between two entities must be available. The standards used in the cellular systems are to be made suitable for global mobile AR services before the commercial utilisation of these novel services can begin to flourish.

One important part – perhaps the most important – in the AR systems is determining the position and orientation of the user. This information is needed for both the accurate registration required to position the virtual objects in the real space and for giving the correct position related information. If the orientation of the user is not determined at all, some context aware applications can still be designed. We see the Wireless Application Protocol (WAP) [6] as a promising and evolving technology to implement mobile AR services in the cellular systems. Before WAP can be fully utilised in mobile AR systems it must support the transmission of the user position and orientation information from the mobile AR terminal (i.e. cellular phone) to the servers in the network. The introduction of open programming environments such as EPOC32 [5] make the development of mobile AR applications easier as most of the applications utilising WAP could not be used through existing microbrowsers, but require a direct connection to the WAP protocol stack.

2. Mobile AR Requirements

As mentioned earlier, accurate position information by itself is not sufficient for many mobile AR applications, which require also the orientation of the user to provide correct annotations to the HMD (Head Mounted Display). The orientation can be attained e.g. from an electronic compass fixed to the HMD. This way the system calibration is easier as the orientation is always relative to the world’s magnetic north. It causes some inaccuracy in the detection of the orientation while indoors, or close to metallic objects. The deviation problem can be solved e.g. by determining the correction based on the location information. As seen in Table 1 some mobile AR applications require both the location and the orientation information of the user.

 Application requirements

Positioning technology approximate

 

Navigation

Context awareness

Multi user dungeon

Tele-presence

GPS

DGPS

Electronic compass

RFID

Cellular network

Passive optical

Spatial

accuracy

1m -10m

5m - 30m

1m – 2m

10m - 100m

1m - 10m

1m - 10m

50m – 1000m

Reacquisition time

1s

2s

1s

0.04s (25Hz)

1s

1s - 2s

0.04s

1s

1s

0.1s-1s

Size

Small

Small

Small

Medium

Small

Medium

Very small

Small

Very small

Medium

Operation area

Outdoors / indoors

Everywhere

Everywhere

Everywhere

Outdoors

Outdoors (in service area)

Everywhere

Very limited

Within network coverage

Preset

Orientation accuracy

2°

5°

1°

0.5°

45° (while walking)

10° (while walking)

0.5° - 2°

1°

Disturbance sensitivity

Medium

Medium

Medium

Low

Medium (structures)

High (structures)

High (metallic objects)

Medium

Low

High (LOS, darkness)

Table 1. Position and orientation accuracy requirements of some mobile AR applications and the accuracy of some possible technologies.

One example application that requires both accurate position information and the orientation of the user (i.e. the line of sight) is navigation. It is also one specific application area WAP is especially good for. Personal navigation services gain added value from additional support information fetched from the network [7]. Such information helps the user by informing the user about the locations and other properties of the destination points, e.g. a selection of restaurants the user can choose from, and their reviews.

We suggest that the position information in the WAP is provided in the form of world co-ordinates, as is done in the Global Positioning System (GPS) [2]. The accuracy should be high enough for future indoor applications, although they will still require much work before they can be made commercially available. We also suggest that the orientation of the user is transmitted as well. This information is required by many mobile AR applications and can also be used to improve many other context aware applications. The orientation should be given related to Earth’s magnetic north, as the compasses do. By utilising such global co-ordinate systems the interoperability of different systems becomes easier. Even though GPS and electronic compass suit best for these co-ordinate formats, other positioning and orientation detection technologies can be used as well.

As a security and privacy issue the user location information is quite controversial. Many users would not like the idea of their location being detected, or even less used without their knowledge. On the other hand almost all feel that in an emergency situations the user location should be detected even if the user is unable to provide it. We suggest that the terminal should be aware of its location and the orientation of the user. This information should be used based on user selection, with one exception. When an emergency call is made (manually or automatically) the location information should be transferred with that call. Otherwise the terminal would provide position and orientation information to the applications on the network based on the preset values of the user’s profile.

3. Example Service Scenarios

Navigation capabilities of mobile terminals are developing all the time. Using WAP one of the simplest navigation service is a simple query: Where am I? A little more complicated service could be a request: How can I find a route to an interesting location? The response could be a list of street names, distances and directions. For example you should go this road 500 meters towards north or towards a visible landmark like a church. Depending on a resolution of the display of the phone, different navigation views can be given. Instead of using a street list, a simple map, where a route is drawn, could be downloaded to the user. More advanced displays like a lightweight augmented reality HMD, would enable the usage of graphical symbols, like arrows, footprints or a virtual guide, pointing to the destination [4].

Navigation is only a tip of the iceberg of possible value-added services of future phones. The implementation of many context-dependent applications becomes possible, after the phone supports positioning. For example tracking of friends, family members and elderly people is possible. Navigation enables a new way to utilise the information in our environment. Context awareness can be used in the implementation of ubiquitous services. For example if a user needs a printer at the airport, the printer can be located using the phone. Advanced entertainment applications will become reality as well. For example very popular Multi-User Dungeon (MUD) games could be played in a real environment, containing virtual presentations of objects and other people.

Ubiquitous computing means that there are computing devices all around us. Devices can support information beyond our normal perceptions or help to control the environment. For example a sensor detecting toxic gases by a busy road, could send a message to a user's phone, if some user is detected to be near the sensor. Depending on the location of the user, the set of available services varies. The location information can be used to download a directory of possible services in that area. The directory can be e.g. a WML page, having links to specific services. Selecting a link, the user can download a virtual user interface of the service. Again depending on the capabilities of the display, the user interface can be a simple textual form or contain more advanced graphical symbols. Position and orientation information can also be applied to place the interface into a right location, if an augmented reality HMD is used.

MUD is an example of a more complicated context-dependent service, where several users are involved. Part of the real world, like a downtown area, could contain a virtual dungeon. Players use different objects, they can find on location basis in the dungeon, to solve the game. A difference to ubiquitous services is that state information has to be maintained between users, resulting to a continuous information stream. The WAP protocol could be used on the top of a packet-switched service like General Radio Packets Service (GPRS) to achieve this. The safety of the players is guaranteed by restricting the area where they can move. For example entering a building or crossing a street could be forbidden.

4. Summary

The rapid development of electronics enables more and more processing power and other capabilities to be integrated into the mobile phones. We see that this development of mobile multimedia devices and services will eventually lead into mobile AR. It basically means that the future mobile terminals are equipped with accurate devices for detecting the user orientation and position, as well as a HMD to present the augmented information onto user’s view of the real world. Many AR applications require high throughput from the network they use. For mobile environments this is provided by the breakthrough of 3rd generation cellular systems. By integrating the positioning service into those upcoming networks we can cost effectively build novel mobile terminals that will be the cellular phones of the future.

WAP is a promising novel technology for creating applications for mobile terminals. By increasing the multimedia capabilities of WAP and by adding means of transferring position and orientation information it becomes a suitable technology for developing commercial mobile AR applications for future mobile phones. The Workshop on Position Dependent Information Systems gives us the opportunity to discuss ways of implementing the positioning services. It is also a forum to discuss whether the orientation of the user should be transmitted as well as the user position. We see it important for the future services to implement the transmission methods of both position and orientation information, and we are prepared to discuss these issues at the workshop.

We have done research in the area of mobile AR for many years. One current research topic of ours is service connectivity in mobile AR. We have also studied protocols for these services as well as several application areas of mobile AR, including personal navigation services. We are prepared to demonstrate a simulation of a mobile AR based personal navigation service for urban areas in which the WAP can be applied, and also present our ideas of utilising WAP for the creation of commercial mobile AR services.

5. Acknowledgements

The authors would like to acknowledge the Nokia Foundation, Jenny and Antti Wihuri Foundation and Infotech Oulu for their financial support on our work.

6. References

[1] Azuma R. T., "The Challenge of Making Augmented Reality Work Outdoors", Proc. of the First International Workshop on Mixed Reality, Japan, March 9. – 10., 1999.

[2] Hoffmann-Wellenhof B., Lichtenegger H, Collins J., "Global Postioning System: Theory and Practise", Springer verlag TELOS, 1997.

[3] Pulli P., Pyssysalo T., Kuutti K., Similä J., Metsävainio J.-P., Komulainen O., "CyPhone – Future Personal Telecooperation Device", Proc. of the XV IFIP World Computer Congress, Austria and Hungary, September 31. – August 4., 1998.

[4] Pyssysalo T., Repo T., Turunen T., Lankila T., Röning J., "CyPhone – Bringing Augmented Reality to Next Genaration Mobile Phones", Submitted to the DARE2000 Conference (http://www.daimi.au.dk/~dare2000).

[5] Symbian, http://www.symbian.com.

[6] The WAP Forum, http://www.wapforum.org.

[7] Turunen T, Lankila T., Pyssysalo T., Röning J., "Realization of Mobile Augmented Reality Based Personal Navigation Services in 3rd Generation Cellular Networks", Submitted to the EuroComm2000 Conference (http://www.comsoc.org/confs/eurocomm/2000).