Driving · 2026-10-01
Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?
Zhihao Sun, Liu Liu, Xinjiang Wang, Haoyi Jiang, Wei Feng, Huiqiang Zhang, Xiaosong Jia, Zhizhong Su, Zuxuan Wu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Ego4WAM studies how egocentric human data properties affect downstream robot learning under a fixed world-action model framework. The authors show that aligned human demonstrations improve out-of-distribution generalization and reduce target-task robot data needs; data duration and task diversity affect different capabilities differently; and video-only supervision remains effective without action labels, helping subsequent video-action training. Closed-loop evaluation is done on real robots and RoboDojo.
Why it matters
Robot learning benefits from scalable egocentric human data, but it’s unclear which data properties and training usage strategies drive robot gains. Ego4WAM disentangles human-robot alignment, coverage (duration vs task diversity), available supervision (action vs video-only), and data usage across pre/mid/post-training, and evaluates via closed-loop robot performance rather than only prediction losses.
Method
- Uses a unified world-action model (WAM) with fixed backbone across main experiments; varies egocentric data alignment, duration, task diversity, action supervision availability, and data usage strategy.
- Organizes training into three stages: world-model pre-training on egocentric video only, joint world+action mid-training on action-annotated data, and downstream robot post-training on embodiment-specific demonstrations (optionally with aligned human data).
- Employs a unified bimanual action space (shared EEF pose+gripper; robot-specific joint signals) and trains action prediction alongside world modeling via conditional flow matching; evaluates with closed-loop success/progress on real robots and RoboDojo.
Abstract (from arXiv)
Egocentric human data provides a scalable source of experience for robot learning, but varies substantially in human-robot alignment, behavioral coverage, and available supervision. Existing work shows favorable scaling with increasing human data, but it remains unclear which data properties drive downstream robot gains and how to use such data throughout the training pipeline. We present a systematic study of egocentric human data with different alignment and supervision under a unified world-action model framework. With the model backbone fixed, we disentangle the effects of human-robot alignment, data duration and task diversity, action supervision, and data usage strategies. We find that aligned human demonstrations substantially improve out-of-distribution generalization and reduce target-task robot data requirements; data duration and task diversity affect downstream capabilities differently; and video-only supervision remains effective without action labels, providing a strong foundation for subsequent video-action training. We validate these findings through closed-loop policy evaluation on both real robots and RoboDojo. Rather than treating data duration as the sole scaling axis, Ego4WAM shows how alignment, task diversity, available supervision, and usage strategy jointly shape the value of egocentric human data for robot learning.
Related papers
- LAPT-Scale: Scaling Embodied Policies with Latent Action Pre-Training on Human Videos
- World Action Learning via Interaction-Centric Spectral Latent Guidance
- Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics
- What 30,000 Hours of Ego-centric Video Does Not Teach
- Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
Ego4WAM: What Matters When Scaling Egocentric Human Data for Robot Learning?
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.40341
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Egocentric human data provides a scalable source of experience for robot learning, but varies substantially in human-robot alignment, behavioral coverage, and available supervision. Existing work shows favorable scaling with increasing human data, but it remains unclear which data properties drive downstream robot gains and how to use such data throughout the training pipeline. We present a systematic study of egocentric human data with different alignment and supervision under a unified world-action model framework. With the model backbone fixed, we disentangle the effects of human-robot alignment, data duration and task diversity, action supervision, and data usage strategies. We find that aligned human demonstrations substantially improve out-of-distribution generalization and reduce target-task robot data requirements; data duration and task diversity affect downstream capabilities differently; and video-only supervision remains effective without action labels, providing a strong foundation for subsequent video-action training. We validate these findings through closed-loop policy evaluation on both real robots and RoboDojo. Rather than treating data duration as the sole scaling axis, Ego4WAM shows how alignment, task diversity, available supervision, and usage strategy jointly shape the value of egocentric human data for robot learning.