Sensorium+: Inklusive Museumslandschaften durch KI-erweiterte Realität

A central component of the project is an AI-driven virtual avatar that functions as a personal museum guide in augmented reality. It communicates using natural language and adaptively presents content tailored to individual user needs. Interaction modalities include not only speech but also gesture and eye-tracking input, facilitating a highly intuitive and accessible user experience.

The platform conveys exhibit information through visual, auditory, and tactile modalities—via targeted sound cues and haptic feedback delivered through specialized AR headset hardware. This multisensory approach creates an immersive, inclusive experience that opens new avenues of access to cultural knowledge, particularly for individuals with visual or auditory impairments.

Emphasis is placed on local processing of all AI functionalities: speech recognition and interaction are executed on-device to ensure data privacy and system efficiency.