AUGMENTED REALITY

Augmented reality is a combination of real and virtual. Interactive virtual objects are overlaid on a video stream in real time, where the real world is registered in 3D. This is also called ”the magic lens metaphor”. In our case the mobile phone is the magic lens, where trough the virtual objects can be seen. There are two types of 3D registration in AR; sensor tracking and optical tracking. In order to use available technology in consumer mobile phones the latter approach will be described. Optical tracking itself is divided in two groups; markerless tracking and marker based tracking. Markerless tracking is certainly the preferred method, but it is computationally expensive, requires information about the environment, and might apply more cameras for accurate results. Marker based tracking uses one or several markers placed in the real world to determine the cameras position and rotation (the transformation matrix) relative each marker. The marker is usually a pattern on a piece of paper. When the orientation of the camera relative the marker is known, virtual objects can be superimposed on each frame of the video stream. The top image illustrates one camera captured frame and the image below illustrates the superimposed frame.

We are using an open source software library for the optical tracking called ARToolKit, which uses single camera 6D tracking that searches for black squares. ARToolKit uses computer vision algorithms to calculate the real camera position and orientation relative to physical markers in real time. In order to use multiple markers different pattern inside the black squares can be used. ARToolKit provides an easy camera calibration routine and it is possible to extend the marker pattern library with your own.