Driving · 2026-11-04
PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations
Travis Davies, Wentao Ma, Lihao Guo, Tengfei Ma, Xiaowen Chang, Jiafeng Xu, Shilong Wang, Yiqi Deng, Shaowen He, Xiaoqi Li, Qian Zhu, Ran Cheng, Hao Dong
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PrimeSim is an action-conditioned, multiview robot world model simulator built by midtraining a pretrained video generator on 30,110 hours of robot demonstrations across 22 platforms and then posttraining it for fast autoregressive simulation with long-horizon stability. It uses unified cross-embodiment action representation, frame-aligned dual-path action conditioning, view-aware interleaved attention, and hierarchical temporal memory.
Why it matters
The authors target core simulator requirements beyond video quality: faithful action following, consistent multiview generation, and coherent long-horizon rollouts. They also report that PrimeSim transfers across platforms quickly and can serve as a learned simulator for GRPO-based policy optimization.
Method
- Midtraining: adapt a pretrained video generator to robot action-conditioned multiview prediction using unified cross-embodiment action representation and frame-aligned dual-path action conditioning.
- Multiview consistency: use view-aware interleaved attention with explicit camera identities for cross-view information exchange while preserving view-specific representations.
- Posttraining: fast-video, slow-memory framework that makes the model autoregressive via causal generation, self-rollout training, and hierarchical temporal memory.
Limitation
Fine-grained local calibration still trails the field.
Abstract (from arXiv)
A robot world model that serves as a simulator must do more than generate plausible videos: it must faithfully follow commanded actions, maintain consistency across camera views, and sustain coherent long-horizon rollouts. We present PrimeSim, an action-conditioned, multiview world model simulator that addresses these challenges through progressive midtraining and posttraining. Starting from a pretrained video generator, we midtrain on 30,110 hours of robot demonstrations spanning 22 platforms. To improve action following, we introduce a unified cross-embodiment action representation and frame-aligned dual-path conditioning to strengthen action-frame correspondence. To maintain multiview consistency, we develop view-aware interleaved attention with explicit camera identities, enabling cross-view information exchange while preserving view-specific representations. We further introduce a fast-video, slow-memory posttraining framework that transforms the bidirectional video generator into an efficient autoregressive simulator. By combining fast causal generation, self-rollout training, and hierarchical temporal memory, PrimeSim mitigates error accumulation from self-generated histories and supports coherent long-horizon simulation. PrimeSim runs at 10.1 FPS, remains stable over long-horizon rollouts on rigid and deformable manipulation tasks, and transfers to held-out platforms within 100 to 200 gradient steps where the pretrained backbone does not. Against five open-source action-conditioned world models it leads four of the five WorldSimProbe suites and posts the best overall EWMBench score, while fine-grained local calibration still trails the field; its rollouts are faithful enough to support GRPO-based policy optimization. We release the model weights, inference code, and training code.
Related papers
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
- OpenWAM: An Open Framework for Composable World-Action Models
- SimForcing: Distilling Simulation Motion Priors into Real-Domain Robot World Models
- RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions
- Robotic Video World Models: A Survey of Applications, Research Challenges, Future Directions
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-04 · Indexed by Pub-AI: 2026-11-04 UTC
PrimeSim: A Generalist World Model Simulator from Large-Scale Cross-Embodiment Demonstrations
First announced by arXiv: 2026-11-04 · Original record ↗
- arxiv
- 2611.04870
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-04 · First detected: 2026-11-04 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.04870v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
A robot world model that serves as a simulator must do more than generate plausible videos: it must faithfully follow commanded actions, maintain consistency across camera views, and sustain coherent long-horizon rollouts. We present PrimeSim, an action-conditioned, multiview world model simulator that addresses these challenges through progressive midtraining and posttraining. Starting from a pretrained video generator, we midtrain on 30,110 hours of robot demonstrations spanning 22 platforms. To improve action following, we introduce a unified cross-embodiment action representation and frame-aligned dual-path conditioning to strengthen action-frame correspondence. To maintain multiview consistency, we develop view-aware interleaved attention with explicit camera identities, enabling cross-view information exchange while preserving view-specific representations. We further introduce a fast-video, slow-memory posttraining framework that transforms the bidirectional video generator into an efficient autoregressive simulator. By combining fast causal generation, self-rollout training, and hierarchical temporal memory, PrimeSim mitigates error accumulation from self-generated histories and supports coherent long-horizon simulation. PrimeSim runs at 10.1 FPS, remains stable over long-horizon rollouts on rigid and deformable manipulation tasks, and transfers to held-out platforms within 100 to 200 gradient steps where the pretrained backbone does not. Against five open-source action-conditioned world models it leads four of the five WorldSimProbe suites and posts the best overall EWMBench score, while fine-grained local calibration still trails the field; its rollouts are faithful enough to support GRPO-based policy optimization. We release the model weights, inference code, and training code.
