Rainer Herpers is Visiting Professor in Canada

The German Academic Exchange Service and the Centre for Advanced Studies of the University of New Brunswick financially support a short-term professorship in computer science for Professor Dr. Rainer Herpers at H-BRS’ Canadian partner university.

Background information (UNB)

Cognitive agents in virtual reality and gaming environments

In most virtual reality simulations (VR) the scenarios presented are no longer static but rather animated. The active entities in VR environments, called "agents", move around and interfere or interact with the user. So far, the simulated behavior of such agents has been scripted. In advanced applications and games, scripted behavior can show a number of variations that are computed on demand through randomization of predefined scripts. This is still the standard approach for agent design, which is applied in almost all available game and serious game application settings.

However, current approaches for creating these agents leads to predictable or unnatural behaviour in the agents. For agents to behave more realistically, the agent’s behavior should reflect the abilities and limitations of actual human beings. A noteworthy example of this concept is in traffic simulations. Although the behavior of all traffic participants is supposed to be predictable and follow well-established rules, real humans tend to take risks and, therefore, might violate these rules from time to time - especially if they find that the risks are considered acceptable under certain conditions (and their violations might not get caught). This will result in behavior changes that result from breaking the rules, which is not randomized but based on an underlying individual risk taking strategy.

The team of the invited expert, Professor Rainer Herpers developed a framework that allows for modelling these sometimes dynamically changing conditions and constraints so that cognitive agents are able to develop individual characteristics, which can even adapt to temporal changes in the environment around them.

To prove this modelling and to provide a particular application context the invited expert has invented an immersive bicycle simulation system called FIVIS a number of years ago. Based on this research project he and his team developed a traffic simulation framework that has been shown more realistic behavior in its agents and can therefore overcome the known shortcomings of existing approaches (e.g., the 4-way stop-sign dilemma).

His current research is focused on virtual perception of human-like virtual agents, which is the logical next step of his previous research. The limitations of humans or better sub-optimal behavior often originate from sensor data limitations, e.g. missing visual information due to occlusions or impaired visual conditions (e.g. fog, darkness or amblyopia) or others (alcohol).

Moreover, a virtual attention strategy has been realized that is integrated into the cognitive agent framework. In this way, the framework is able to model and compute sub-optimal, but more realistic, agent behavior by using impaired sensory information.

Research Project: EpicSave

In an on-going research project called EpicSave his team has started to embed and evaluate these cognitive agent concepts within professional training scenarios for paramedics in uncommon and rare situations such as anaphylactic shocks. Paramedic training requires a high degree of communication between the different actors or agents under high stress levels and distracting conditions, so that virtual cognitive agents will allow for more realistic virtual but repetitive training scenarios applying VR technologies such as head-mounted displays (among others).

During his stay at UNB, the invited professor will import his knowledge to the expertise of existent faculty members of the Faculty of Computer Science working in the area of game technology, modelling, and human computer interaction. He will work with Dr. Scott Bateman who has established the Human-Computer Interaction Lab within the Faculty of Computer Science. Dr. Bateman's team is familiar with game design and the design of interactive game environments for training and other "serious game" applications. Further, Dr. Bateman has a background in human-performance and perception, which will provide important expertise when creating new, realistic behavior models based upon the "perception" of cognitive agents.

The research stay should be used to develop strategies how to transform and apply these cognitive agent approaches to serious game scenarios in a more generalized way. Moreover, generic evaluation strategies should be developed to show the abilities of these simulation concepts. New graduate students will be recruited and integrated into the new research initiatives. In the ideal case, new dual master degree students of both institutions will take advantage of this research stay and might accompany the invited professor.

Moreover, the computation of virtual perception strategies of cognitive agents is quite computationally expensive, which in addition will even scale up as soon as multiple agents need to be integrated in VR simulation environments. For that, high performance computation approaches would be beneficial to speed up the computation. In this area of research another UNB faculty member is considered to collaborate with the invited scientist, which will bring in his expertise in GPGPU computing and other parallelization techniques. Dr. Kenneth Kent has extensive experience in high-performance computing and leads the Centre for Advanced Studies - Atlantic. A recent AIF project of the Centre, led by Dr. Kent, examined improving the performance of the J9 Java Virtual Machine, IBM’s flagship platform for Java applications. Dr. Kent’s research has been quite successful with over 100 publications and roughly 18 invention disclosures/patents.

The home institution of the invited faculty member (H-BRS) is already intensively collaborating with FCS at UNB for many years. Two dual undergraduate degree programmes, one dual master degree programme and a framework for student and faculty exchanges has been successfully implemented. In particular the dual master programme showed a considerable success in the past years demonstrated in numerous joint scientific publications. At H-BRS a new master programme in Visual Computing and Games Technology is under development. It is planned to use the research stay also to identify possible collaboration options in this new graduate programme