Practice makes perfect. A truly agentic robot shouldn’t just know what to do. It should also figure out how to get better.
What if a robot could watch a demonstration, figure out what it needs to practice, and learn how to get better through practice?
Introducing RPG: Guided Self-Improvement for Embodied Agents, a real-to-sim-to-real pipeline that turns real-world demonstrations into targeted practice.
RPG reconstructs practice tasks from demonstrations, uses demo videos to guide skill improvement, and continually updates a shared skill system from failures.
On real robots, RPG achieves 30/30 successes across 3 tasks. In simulation, it reaches 95.0% success across 22 manipulation tasks after 15 rounds of self-improvement.
Demo → Reconstruct → Guided Practice → Self-Improve → Go Real
What if a robot could watch a demonstration, figure out what it needs to practice, and learn how to get better through practice?
Introducing RPG: Guided Self-Improvement for Embodied Agents, a real-to-sim-to-real pipeline that turns real-world demonstrations into targeted practice.
RPG reconstructs practice tasks from demonstrations, uses demo videos to guide skill improvement, and continually updates a shared skill system from failures.
On real robots, RPG achieves 30/30 successes across 3 tasks. In simulation, it reaches 95.0% success across 22 manipulation tasks after 15 rounds of self-improvement.
Demo → Reconstruct → Guided Practice → Self-Improve → Go Real
1:44 8 50 15 222 164K 137
Mahi Shafiullah 🏠🤖
Jason Ma
Jim Fan