IFML Seminar: 10/30/26 - What Can Robots Learn from Failure and How

Abstract: Robot manipulation has advanced rapidly by imitating successful demonstrations, but robots still fail constantly. Sadly, researchers mostly do not record or share these failures, leaving a rich source of data and research questions untapped. This talk asks: What can we learn from failure, and how?
The first part of the talk will focus on improving pre-trained foundation policies. Such foundation policies can often transfer to novel tasks, which makes them exciting starting points for general-purpose control methods, but rarely succeed reliably enough to be useful in practical settings. Can we use the policy’s failures into success? To address this, we can use guidance methods which adjust a pre-trained robotic control policy based on a trained failure classifier. I will show how guidance can achieve significant improvements in control accuracy, improving accuracy by up to 50 percentage points on held-out tasks, across a range of pre-trained policies.

Training guidance classifiers at scale requires failures, which raises the second question of the talk: where should good failure data come from? Over the past year, we have addressed this question with researchers at the University of Texas at Austin and UC Berkeley. Together, we coordinated a consortium of over 60 labs and collected Oopsie-Data, the first large-scale crowd-sourced dataset of realistic robot manipulation failures. Oopsie-Data opens up a wide range of research opportunities in robotics, and it lets us tackle core questions in the data science of failure: how failure should be described, labeled, and used.

Talk based on joint work with Maria Attarian, Yifan Ruan, Arpit Bahety, Paul Zhou, Ren Wang, Roberto Martín-Martín, Peter Stone, and Amy Zhang.

Bio: Dr. Claas Voelcker is a postdoctoral researcher at the University of Texas at Austin. His research interests lie in reinforcement learning and robotics. He received a PhD from the University of Toronto, where he was advised by Igor Gilitschenski and Amir-massoud Farahmand.