Driving · 2026-10-08

EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution

Python Song, Zhixuan Liang, Kelsey Fu, Mengdi Wang, Junfeng Yang, Shilong Liu

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

EmbodiedRSI is a self-evolving agentic robot harness that uses a Fast-Slow dual-system and a Hypothesis Graph to decide which physical experiment to run next via Value-of-Information. Outcomes drive joint Code-Skill Co-Evolution and Reward-Grounded Memory Learning. With the robot foundation model frozen, it improves generalization on RoboCasa365 and LIBERO-Pro and transfers zero-shot from simulation to a real robot.

Why it matters

The authors address inefficient learning from costly physical trials in self-evolving robotics harnesses. EmbodiedRSI explicitly allocates experiments using expected uncertainty reduction, refines code/skills jointly using matched trials, and learns a hierarchical memory policy based on later fast-system improvement—aiming to convert physical evidence into reusable improvements for unseen scenes and tasks.

Method

Fast-Slow dual-system: a Fast System maintains hypotheses in a Hypothesis Graph and selects the next physical experiment using Value-of-Information Experiment Selection; a Slow System builds Hierarchical Memory and trains Reward-Grounded Memory Learning via PPO.
Code-Skill Co-Evolution: for each candidate, the harness evaluates code-only, skill-only, and joint execution from the same initial states, using evidence to update code-skill relations in the Hypothesis Graph.
Reward-Grounded Memory Learning: the Slow System learns memory actions (Retain/Merge/Abstract/Compress/Forget/Skip) according to downstream value measured by later task improvement, exploration-uncertainty reduction, and physical-experiment efficiency.

Limitation

Code-as-policy and agentic harness methods learn executable behavior from physical trajectories without updating the robot foundation model’s parameters. Future work could use trajectories selected by the harness for online model updates, allowing code, skills, and the model to improve together.

Abstract (from arXiv)

Robot foundation models provide strong visuomotor control, yet their performance can degrade when object positions or task instructions change. Further improvements often require post-training on substantial robot data, which can be costly to collect through methods such as teleoperation. Agentic harnesses can adapt around the model, but current self-evolving harnesses use robot trials inefficiently when deciding which code and skill changes to pursue. We introduce EmbodiedRSI, a self-evolving agentic harness that autonomously decides where to explore next and turns the resulting physical interaction into improved code and skills. EmbodiedRSI realizes this through a Fast-Slow Dual-System Architecture, in which competing code and skill hypotheses are maintained in a Hypothesis Graph. Value-of-Information Experiment Selection chooses physical experiments that can distinguish these hypotheses. Their outcomes guide Code-Skill Co-Evolution. The Slow System builds Hierarchical Memory, and Reward-Grounded Memory Learning selects effective memory according to their value for later Fast-System improvement. On RoboCasa365, EmbodiedRSI reaches 77.0% overall success and 71.3% on Composite-Unseen, compared with 40.1% for the best baseline. EmbodiedRSI also reaches 86.8% overall success on LIBERO-Pro. Beyond benchmark performance, EmbodiedRSI transfers zero-shot to real-world robot, achieving 71.3% overall success across multiple challenging tasks.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC

preprint