Driving · 2026-10-14

SEAR: Self-Evolving Agentic Robotics

Xinyi Wang, Bangzheng Li, Hanyu Wang, Lin Zhang, Andrea Yaoyun Cui, Zhenggang Tang, Shyamal Buch, Dejia Xu

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

SEAR (Self-Evolving Agentic Robotics) is a framework and benchmark for testing whether LMs improve robots through self-evolution, disentangling true learning from effects like resampling, luck, and privileged simulator state. It runs agents in three modes (code-as-policy, agent-as-policy, auto-research) and evaluates across 330 simulated and five physical tasks using seven frontier LMs, with retained “libraries” carrying skills and memories across tasks.

Why it matters

The authors argue that rising success alone is ambiguous and can be explained by trying more often, luck, bigger budgets, or simulator artifacts. SEAR adds paired controls (best-of-3, no-library, sealed judge, luck repetitions, budget matching) and multiple generalization tiers to test whether retained experience helps on new tasks, new simulators, and on real robots.

Method

Three agent modes: code-as-policy (LM writes a program policy), agent-as-policy (LM acts in the control loop), and auto-research (collects rollouts, curates data, fine-tunes a VLA checkpoint, and selects what to try next).
Retained experience is stored as a versioned library of code skills and written memories carried between tasks; simulation uses code-as-policy and auto-research, and hardware uses agent-as-policy.
The benchmark adds controls and evaluation: budget-matched best-of-3 restarts vs one long session, runs with no library (vanilla), a sealed judge that grades frozen policies on held-out initial states, and three repetitions to expose luck; generalization tiers T1–T5 measure distance from practice.

Abstract (from arXiv)

Frontier language-model agents can now improve robots on their own: they write control code, test it, and keep what works as skills and memories for the next task. We introduce SEAR (Self-Evolving Agentic Robotics), a framework and benchmark built around two questions. Is the agent really learning, or just trying more often? And does what it learns still help on new tasks, in a new simulator, and on a real robot? The framework runs an agent in three modes: it writes code that becomes the policy (code-as-policy), acts live in the control loop (agent-as-policy), or works as a researcher that fine-tunes a vision-language-action model (auto-research). The benchmark adds controls that separate retained experience from repeated attempts, familiar tasks, privileged simulator state, larger budgets, and luck. Across seven frontier language models, 330 simulated and five physical tasks, and over four years of agent runtime, we highlight three findings. First, the learning curve of a robotic agent depends both on the overall time budget and the task difficulty relative to model capability. Harder tasks make longer sessions worthwhile; on easier tasks, re-rolling short attempts does just as well. Second, control code and shared routines transfer across tasks and simulators, while task-specific solutions do not. On a real robot, we observe a performance gap between agent-designed practice and novel human-designed scenes. Third, different models evolve more effectively in different ways. Some do best acting live, others writing code first. And a text-only model keeps pace with a comparable multimodal one: perception helps with error assessment, but agentic reasoning and tool use can matter more. We release the benchmark and codebase to support future research.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC

preprint