Driving · 2026-10-07
SMART: Zero-Shot Sim-to-Real Articulated Object Manipulation via Large-Scale Synthetic Pretraining
Jicong Ao, Shuhan Jiang, Yuling Zhong, Yanwen Liu, Yuhan Gao, Jiangyuan Zhao, Yang Zhang, Shiqiang Zhu, Chenjia Bai, Xuelong Li
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SMART introduces SMART-Sim, an articulation-aware simulation platform, and an agentic/distributed synthesis pipeline to generate SMART-Data: over 1M synthetic demonstrations across 44 atomic task types, 5 robot setups, and 2,507 articulated objects. A VLA model pretrained on SMART-Data achieves competitive simulation benchmark performance and reports zero-shot sim-to-real transfer plus scalable performance for real-world articulated-object manipulation tasks.
Why it matters
The authors target contact-rich articulated-object manipulation, where real-world data collection is costly and simulation datasets previously covered limited articulated categories or lacked explicit part-level semantics and articulation constraints. By building articulation-aware simulation and large-scale synthetic pretraining, SMART aims to provide scalable supervision for VLA learning and sim-to-real transfer.
Method
- Develop SMART-Sim (built on Isaac Lab) with articulation-aware designs for asset annotation, motion generation, skill definition, and domain randomization.
- Use agentic task generation plus a distributed data synthesis system to scale trajectory synthesis into SMART-Data.
- Pretrain a vision-language-action (VLA) model on SMART-Data and evaluate on simulation benchmarks plus zero-shot sim-to-real and scaling experiments.
Abstract (from arXiv)
The ability to interact with articulated objects is essential for embodied intelligent systems, but collecting large-scale real-world demonstrations for these interactions remains challenging due to the precise contact and constraint-following motions involved. Although simulation provides a promising alternative, existing synthetic data efforts cover limited articulated-object categories, while general-purpose synthesis pipelines lack explicit designs for part-level semantics and articulation constraints, hindering agentic task generation and scalable synthesis of high-quality articulated-manipulation demonstrations. To bridge this gap, we introduce SMART, a scalable system leveraging large-scale Synthesized Manipulation demonstrations for ARTiculated-object manipulation. At its core, we develop SMART-Sim, a simulation platform with articulation-aware design that enables effective task generation and efficient demonstration collection. Building on SMART-Sim, we apply agentic task generation and design a scalable distributed synthesis system, using them to synthesize SMART-Data, comprising over 1M demonstrations across 44 atomic task types, 5 robot setups, and 2,507 articulated objects. The vision-language-action (VLA) model pretrained on SMART-Data shows competitive performance on simulation benchmarks and achieves zero-shot sim-to-real transfer and scalable performance in real-world articulated-object manipulation tasks. This highlights the potential of synthetic demonstrations in providing effective and scalable supervision for improving VLA model performance in contact-rich articulated-object manipulation.
Related papers
- AutoRSR: Automated Real-to-Sim-to-Real Data Generation for Scalable Robot Learning
- Real2Gym: Building Gyms from Videos, Bringing Skills to Robots
- CASTER: Formalizing Zero-Shot Manipulation from Synthetic Videos as Constrained Optimization
- EmbodiedSmith: Scaling Embodied Data through Recursive Self-Improvement Flywheel in Simulation
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
SMART: Zero-Shot Sim-to-Real Articulated Object Manipulation via Large-Scale Synthetic Pretraining
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.07652
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
The ability to interact with articulated objects is essential for embodied intelligent systems, but collecting large-scale real-world demonstrations for these interactions remains challenging due to the precise contact and constraint-following motions involved. Although simulation provides a promising alternative, existing synthetic data efforts cover limited articulated-object categories, while general-purpose synthesis pipelines lack explicit designs for part-level semantics and articulation constraints, hindering agentic task generation and scalable synthesis of high-quality articulated-manipulation demonstrations. To bridge this gap, we introduce SMART, a scalable system leveraging large-scale Synthesized Manipulation demonstrations for ARTiculated-object manipulation. At its core, we develop SMART-Sim, a simulation platform with articulation-aware design that enables effective task generation and efficient demonstration collection. Building on SMART-Sim, we apply agentic task generation and design a scalable distributed synthesis system, using them to synthesize SMART-Data, comprising over 1M demonstrations across 44 atomic task types, 5 robot setups, and 2,507 articulated objects. The vision-language-action (VLA) model pretrained on SMART-Data shows competitive performance on simulation benchmarks and achieves zero-shot sim-to-real transfer and scalable performance in real-world articulated-object manipulation tasks. This highlights the potential of synthetic demonstrations in providing effective and scalable supervision for improving VLA model performance in contact-rich articulated-object manipulation.