Department of Computer Science
How can multisensory stimuli be used to elicit a complex response in VR?
In cooperation with the Institute of Visual Computing (IVC) and the Cybernetics and Reality Engineering Laboratory (CARE Lab), headed by Prof. Kiyoshi Kiyokawa, we will use multisensory stimuli in an attempt to trigger complex emotional responses such as the feeling of awe in VR. Awe is a positive, self-transcending emotion that makes people feel like they are part of something bigger than themselves. It occurs when people experience something grand or sublime, such as natural wonders or impressive architecture. Since awe contributes to greater life satisfaction, the research team believes that effectively experiencing awe in VR could increase well-being. The use of multisensory stimuli, e.g., audiovisual stimuli of impressive environments complemented by various tactile stimuli such as vibration or heat stimuli, or stimuli related to the sense of balance, will be used to specifically evoke the feeling of awe in virtual environments.
In April/May 2023, a small delegation consisting of Assistant Professor Monica Perusquía-Hernández and Dr. Felix Dollack from Japan came to the H-BRS for the first time to work on preparations for the first underwater test. These tests, supported by Volker Heemann (Stadtwerke Osnabrück) were performed together in the Nettebad of the Stadtwerke Osnabrück. Here, commercial underwater HMDs with different VR scenarios are in use, which were tested in use and function as part of the research.
After the first successful tests and findings, the H-BRS research team, consisting of Prof. Dr. André Hinkenjann, Prof. Dr. Ernst Kruijff (project leader), Dr. Alexander Marquardt (deputy project leader), student assistant Melissa Meilinger and student assistant Marvin Lehnort, visited NAIST in June 2023. Beforehand, Hinkenjann and Kruijff were able to attend a Shonan workshop on "Augmented Perception" organized by Yuta Itoh (The University of Tokyo, Japan), Kai Kunz (Keio University, Japan) and Jason Orlosky (Augusta University, USA & Osaka University, Japan).
One specific VR scenario tested in the pool in conjunction with the underwater HMD (head-mounted display) was the view from space to Earth - the so-called overview effect - which is a famous example of a sense of awe, as the feeling of "infinite space" is said to be particularly palpable here. Multisensory technology plays a role here in that underwater stimuli exert an influence on the sense of balance to simulate a feeling of weightlessness, which of course is particularly congruent with the visual stimuli of the VR scene.
The special thing about the cooperation with NAIST is that it operates its own small pool facility for research purposes. Marquardt used the pool to optimize the underwater prototypes developed in this project based on the findings from the visit to Osnabrück and to test them in the controlled pool environment. In Virtual Reality Society of Japan (VRSJ) 2023, his study has now been published and can be read here:
