Advanced Topics in AI and Robotics

Abstract

Humans use non-verbal behaviors to signal their intent, emotions, and attitudes in human-human interactions. Embodied conversational agents (ECA), therefore, need this ability as well in order to make interaction pleasant and efficient. An essential part of non-verbal communication is gesticulation. The task of hand gesture generation has been initially approached mainly by rule-based methods, but current state-of-the-art methods are data-driven, and we continue this line of research.

In this talk, Taras will present two papers addressing the two components of the task: generation and evaluation of co-speech gestures for embodied conversational agents:

1) Speech2Properties2Gestures, a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures.

2) GENEA Challenge 2022, the second gesture-generation challenge, intended as a benchmark of data-driven automatic co-speech gesture generation

Short Bio

Dr. Taras Kucherenko is currently a Research Scientist at Electronic Arts. Recently he finished a Ph.D. at the KTH Royal Institute of Technology in Stockholm. His research is on machine learning models for non-verbal behavior generation, such as hand gestures and facial expressions.

For his research papers, he received ICMI 2020 Best Paper Award and IVA 2020 Best Paper Award.

Taras was also the main organizer of The GENEA (Generation and Evaluation of Non-verbal Behavior for Embodied Agents) Workshop and Challenge in 2020, 2021, 2022, and 2023.

Papers relevant for the talk

• Simba Nyatsanga, Taras Kucherenko, Chaitanya Ahuja, Gustav Eje Henter, and Michael Neff. (2023). A Comprehensive Review of Data-Driven Co-Speech Gesture Generation. In Computer Graphics Forum (EuroGraphics 2023)
• Taras Kucherenko, Rajmund Nagy, Michael Neff, Hedvig Kjellström, and Gustav Eje Henter. Multimodal analysis of the predictability of hand-gesture properties. 21st International Conference on Autonomous Agents and Multiagent Systems (AAMAS). 2022
• Youngwoo Yoon*, Pieter Wolfert*, Taras Kucherenko*, Carla Viegas, Teodor Nikolov, Mihail Tsakov, and Gustav Eje Henter. 2022. The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation. In Proceedings of the ACM International Conference on Multimodal Interaction (ICMI ’22). ACM.

The lecture will be held in English and is aimed at students and staff of the H-BRS. Interested parties are cordially invited.