Driving · 2026-10-16
Cross-Embodiment Causal Transfer from Human Videos to Robot Manipulation
Xingyu Zhang, Yingchao Li, Peizheng Guo, Jie Hu, Changwen Zheng, Wenwen Qiang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
CBST (Causal-Beam-Guided Subtask Transfer) transfers cross-embodiment knowledge from human manipulation videos to robot world-action models. It learns manipulation-related representations from human videos, grounds them into task-level semantic variables using robot demonstrations, then uses causal-beam constraints to select context-dependent retained dependencies. These semantic conditions let the robot generate actions from its own observations without human–robot motion correspondence. Experiments show improved task success over world-action model baselines.
Why it matters
Collecting diverse real-robot experience for world-action models is costly. This work targets what to transfer from abundant human manipulation videos across embodiment differences (morphology/action spaces/control mechanisms) by using causal-beam constraints to retain effect-relevant contextual dependencies for local task effects, enabling action generation without direct motion correspondence.
Method
- Pretrain a visual–temporal encoder on human videos via action-category classification (ResNet18–GRU), then adapt it with robot demonstrations to estimate task-level semantic variables for local manipulation subtasks.
- Instantiate a candidate manipulation SCM from shared manipulation templates and use learned contextual causal beams: a selector chooses compatible retained parent subsets/assignments under sustenance and responsiveness constraints relative to reference mechanisms.
- Render the selected subtask information as language conditions for a frozen world–action model (WAM), which generates action chunks and predicts future observations from the robot’s own observations.
Abstract (from arXiv)
Scaling robot manipulation models requires large amounts of interaction data, yet collecting diverse real-robot experience remains costly. Human manipulation videos, in contrast, provide abundant and diverse task experience that could potentially benefit robot learning. However, differences in morphology, action spaces, and control mechanisms make human motions unsuitable as direct supervision for robot actions, raising a scientific question: what knowledge in human videos should be transferred across embodiments to robot manipulation? To address this question, we analyze subtask transfer through the lens of causal beams, focusing on which contextual conditions and interaction dependencies should be retained to support local task effects. Based on this perspective, we propose Causal-Beam-Guided Subtask Transfer (CBST). CBST learns manipulation-related representations from human videos and grounds them in task-level semantic variables using robot demonstrations. Within a predefined manipulation model, causal-beam constraints guide contextual information selection by requiring retained assignments to sustain local mechanism outputs while preserving nontrivial responsiveness to retained inputs. The selected semantic information conditions a world-action model, allowing the robot to generate actions from its own observations without requiring human-robot motion correspondence. Experiments on robot manipulation benchmarks show improvements in task success over the corresponding world-action model baselines.
Related papers
- Learning Transferable World-Action Models from Task-Paired Human Videos
- World Action Learning via Interaction-Centric Spectral Latent Guidance
- RoboChrono: A Real Robot Benchmark for Streaming Task Understanding
- Cross-Embodiment Robot Foundation World Models with Latent Actions
- Causal Analysis of Robot Foundation Models Enables Efficiency Gains
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-15 · Indexed by Pub-AI: 2026-10-16 UTC
Cross-Embodiment Causal Transfer from Human Videos to Robot Manipulation
Preprint posted: 2026-10-15 · Original record ↗
- arxiv
- 2610.21798
Explicit source versions (1)
- v1 · 2026-10-15
- Preprint available
Source date: 2026-10-15 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Scaling robot manipulation models requires large amounts of interaction data, yet collecting diverse real-robot experience remains costly. Human manipulation videos, in contrast, provide abundant and diverse task experience that could potentially benefit robot learning. However, differences in morphology, action spaces, and control mechanisms make human motions unsuitable as direct supervision for robot actions, raising a scientific question: what knowledge in human videos should be transferred across embodiments to robot manipulation? To address this question, we analyze subtask transfer through the lens of causal beams, focusing on which contextual conditions and interaction dependencies should be retained to support local task effects. Based on this perspective, we propose Causal-Beam-Guided Subtask Transfer (CBST). CBST learns manipulation-related representations from human videos and grounds them in task-level semantic variables using robot demonstrations. Within a predefined manipulation model, causal-beam constraints guide contextual information selection by requiring retained assignments to sustain local mechanism outputs while preserving nontrivial responsiveness to retained inputs. The selected semantic information conditions a world-action model, allowing the robot to generate actions from its own observations without requiring human-robot motion correspondence. Experiments on robot manipulation benchmarks show improvements in task success over the corresponding world-action model baselines.