KEROL

Work on robot skills, including recent robot foundation models, has demonstrated that learning-based skills can produce quite flexible robot behaviour; however, the generalisation and overall robustness of learned skills is often unclear in advance, which means that the possibility for unsafe behaviour during execution also cannot be excluded. To use learned skills responsibly, we need to, on the one hand, be able to evaluate those extensively and, on the other hand, use additional runtime execution monitoring techniques so that a robot can recognise and recover from unforeseen execution anomalies and failures.

In KEROL, we aim to make a small step towards enabling these properties, by concretely focusing on two aspects:

1. Data collection and annotation for robot skill learning: The first goal of KEROL is to develop a standardised interface that should simplify the process of triggering the execution of various manually-specified or learned robot skills — potentially under anomalous conditions — and the subsequent annotation of the collected data. The particular intention here is to simplify the annotation of failures in the execution data so that this information can be subsequently utilised for skill (re)training, but also for learning failure analysis models.
2. Simulation-based skill evaluation: KEROL's second goal is to develop a simulation-based environment in which learned skills can be evaluated under various conditions. This has the objective of testing the robustness of the skills under different conditions as well as uncovering cases in which the skills are likely to fail. The ultimate goal is to use this information to guide any subsequent data collection so that the skill model can be improved and the epistemic uncertainty can be reduced accordingly.

The activities in KEROL are performed in the context of domestic tasks, such as coffee preparation.