“HoloLens" improves Augmented Reality (AR)

The glasses project three-dimensional images into the user’s real environment. They can then operate them on an interactive basis. It is also smaller than the previous model. What is new about the project is that users of these ‘mixed reality’ glasses can receive both audio and sensory stimulus. The German Research Foundation (DFG) is sponsoring the project with some EUR 520,000 over three years.

Current AR glasses have a relatively small field of vision, but the volume of information increases with the complexity of the applications. Experience shows that the wealth of information needs to be reduced in order to be processed profitably without compromising on comprehensibility or user-friendliness.

The team led by Dr Ernst Kruijff, Interim Professor for Interactive Environments at H-BRS, aims to change this. The new approach combines augmented reality with multisensory feedback. For the scientists, this means translating some visual stimuli and other information into sensory or audio content.

One question asked by the research in this area is whether users can process information with multisensory feedback better. This information could be passed on by means of a slight vibration, like a slight itch on the head. The vibration can come from different sides to inform the user which direction more detailed information can be found in. What is known as ‘eye-tracking’ is used to determine which information is to be translated. Eye-tracking calculates the viewing angle of the user in order to understand what objects they are focussing on and, as a result, give them the appropriate information.

“While the potential applications of AR largely depend on visual perception, there are still very few research projects on the subject,” explains Dr. Ernst Kruijff. “The project is an important contribution to fundamental research in the field of multisensory presentation of information in AR systems. The anticipated results are of real interest for a wide range of applications and systems,” continues Kruijff.

Kruijff’s team aims to test the new approach both under controlled laboratory conditions and in the field: In Autumn 2017, the research team will be travelling to Columbia University/New York and Mississippi State University for this purpose. In their luggage will be the new prototype of the AR glasses with eye-tracking.