Faseeh Ahmad's PhD defence

Thesis title: Towards Self-Reliant Robots: Skill Learning, Failure Recovery, and Real-Time Adaptation

Author: Faseeh Ahmad, Department of Computer Science, Lund University

Faculty opponent: Professor Lazaros Nalpantidis, Technical University of Denmark (DTU), Denmark

Examination Committee:

• Associate Professor Casper Schou, Aalborg University, Denmark
• Professor Karinne Ramirez-Amaro, Chalmers University of Technology
• Dr. Mikael Norrlöf, ABB Robotics R&D Västerås
• Deputy: Senior Lecturer Yiannis Karayiannidis, Lund University

Session chair: Senior Lecturer Michael Doggett, Lund University

Supervisors:

• Professor Volker Krueger, Lund University
• Professor Jacek Malec, Lund University

Location: E:1406, E-huset, Klas Anshelms väg 10/Ole Römers väg 3, Lund

Here is a link to download the thesis at LU Research Portal

Abstract

Robots operating in real-world settings must manage task variability, environmental uncertainty, and failures during execution. This thesis presents a unified framework for building self-reliant robotic systems by integrating symbolic planning, reinforcement learning, behavior trees (BTs), and vision-language models (VLMs).

At the core of the approach is an interpretable policy representation based on behavior trees and motion generators (BTMGs), supporting both manual design and automated parameter tuning. Multi-objective Bayesian optimization enables learning skill parameters that balance performance metrics such as safety, speed, and task success. Policies are trained in simulation and successfully transferred to real robots for contact-rich manipulation tasks.

To support generalization, the framework models task variations using gaussian processes, enabling interpolation of BTMG parameters across unseen scenarios. This allows adaptive behavior without retraining for each new task instance.

Failure recovery is addressed through a hierarchical scheme. BTs are extended with a reactive planner that dynamically updates execution policies based on runtime observations. Vision-language models assist in detecting and identifying failures, and in generating symbolic corrections when tasks are predicted to fail.

The thesis concludes with a discussion of future work, including (1) using vision-language-action (VLA) models or diffusion policies to generate new skills on the fly from multimodal inputs, and (2) extending the reactive planner with proactive failure prediction to anticipate and prevent execution errors before they occur. Together, these directions aim to advance robotic systems that are more robust, adaptable, and autonomous.