Robotics · 2026-10-05
Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation
Xincheng He, Siyu Ma, Chang Yu, Yunuo Chen, Yanjia Huang, Ying Nian Wu, Yin Yang, Chenfanfu Jiang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Skill2Real learns transferable, executable robot skills in simulation via an asymmetric Proposer–Verifier–Governor (PVG) loop and transfers frozen Cerebellum (local manipulation) and Brain (task-level composition) skills to real robots through a shared code-based API. The authors report LIBERO-Pro Long zero-shot transfer improvements and real UR5e results without real-world task-policy fine-tuning or skill-memory updates.
Why it matters
The authors target the sim-to-real gap for long-horizon manipulation by separating privileged simulation supervision (for diagnosing and validating updates) from what is used at deployment (public observations + API semantics). This aims to preserve task-level knowledge in a reusable hierarchy that can transfer without training a new representation or adapting policies on the real robot.
Method
- Learns a two-level hierarchy: Stage I trains and freezes Cerebellum local manipulation skills; Stage II freezes it and trains Brain skills to compose long-horizon programs with planning and recovery.
- Uses an asymmetric PVG loop in simulation: a Proposer proposes updates from public observations and API returns, a Verifier diagnoses using privileged simulation evidence, and a Governor admits updates supported by validation rollouts.
- Transfers only frozen skill memories to deployment through a shared code-based policy interface; deployment keeps Proposer only and does not fine-tune task policies or update skill memories.
Limitation
Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively.
Abstract (from arXiv)
Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embodiment. We introduce Skill2Real, an agentic policy framework that learns executable skills through a shared application programming interface (API). A Proposer-Verifier-Governor (PVG) loop uses privileged simulation evidence to diagnose outcomes and validate updates, while keeping learned skills grounded in public observations and API semantics. The Cerebellum first acquires local manipulation skills; the Brain then learns task-level composition with the Cerebellum frozen. Both memories transfer to the real robot without task-policy fine-tuning or skill-memory updates. As GPT-5.6 Sol learns skills on LIBERO-90, evaluating each frozen checkpoint with GPT-6 Astra raises LIBERO-Pro Long success from 2.0% to 56.3%, without training on Pro Long. Independent Robosuite training reaches 85.1% and 89.4% mean success with Sol and Opus 5 across seven tasks, respectively. Frozen Sol-trained LIBERO-90 skills achieve 78.75% mean completion across four real-world manipulation tasks with Astra. Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively. These results support learning and transferring a hierarchy of executable skills through a common robot interface.
Related papers
- SkillWeaver: Agentic Exploration over Neural Interaction Skills for Scalable Robot Data Generation
- RoboBridge: A Self-Evolving Embodied Agent Framework for Sim-to-Real Transfer
- Real2Gym: Building Gyms from Videos, Bringing Skills to Robots
- Explore, Execute, Evolve: A Skill Acquisition and Reuse Loop for Embodied Agents
- EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-02 · Indexed by Pub-AI: 2026-10-05 UTC
Skill2Real: Agentic Skill Learning for Zero-Shot Sim-to-Real Robot Manipulation
Preprint posted: 2026-10-02 · Original record ↗
- arxiv
- 2610.02788
Explicit source versions (1)
- v2 · 2026-10-14
- Preprint available
Source date: 2026-10-02 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
- Explicit source version detected · v2
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
Abstract · arxiv
Transferring robotic skills from simulation to reality requires task knowledge that remains usable across differences in perception, dynamics, and embodiment. We introduce Skill2Real, an agentic policy framework that learns executable skills through a shared application programming interface (API). A Proposer-Verifier-Governor (PVG) loop uses privileged simulation evidence to diagnose outcomes and validate updates, while keeping learned skills grounded in public observations and API semantics. The Cerebellum first acquires local manipulation skills; the Brain then learns task-level composition with the Cerebellum frozen. Both memories transfer to the real robot without task-policy fine-tuning or skill-memory updates. As GPT-5.6 Sol learns skills on LIBERO-90, evaluating each frozen checkpoint with GPT-6 Astra raises LIBERO-Pro Long success from 2.0% to 56.3%, without training on Pro Long. Independent Robosuite training reaches 85.1% and 89.4% mean success with Sol and Opus 5 across seven tasks, respectively. Frozen Sol-trained LIBERO-90 skills achieve 78.75% mean completion across four real-world manipulation tasks with Astra. Removing the Verifier or Governor during LIBERO-90 training lowers final Pro Long success by 17.3 and 13.3 percentage points, respectively. These results support learning and transferring a hierarchy of executable skills through a common robot interface.