Driving · 2026-10-28
World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer
Guoning Wu (World Agents), Zijian Cai (World Agents), Yuhang Zhang (World Agents), Ying Liu (World Agents), Yongbin Zheng (World Agents)
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
World Dreamer v5-lite is a multi-robot world-action model built around a shared world state (ϕ) in workspace coordinates. Calibrated features from a pretrained video encoder populate ϕ, while a vision-language model provides scene context and one query per robot. Queries exchange pose-aware information, but action streams remain separate. Masked-view cooperative training withholds robot views while keeping action targets.
Why it matters
The authors target multi-robot control where distributed cameras have no direct correspondence across views and where robots must coordinate under incomplete observations. Their shared, calibrated world state provides a fixed spatial reference as cameras vary, and masked-view training is designed to force policies to use observations from teammates rather than relying only on each robot’s own cameras.
Method
- Builds a shared world state ϕ from calibrated multi-view video features by projecting patch tokens into workspace coordinates and writing them to XY, XZ, and YZ planes with a visibility-gated update.
- Uses a vision-language model with one learnable query per robot; queries read the shared ϕ and exchange pose-aware information via agent interaction attention, while each robot keeps a separate action stream.
- Introduces masked-view cooperative training that withholds selected camera views from both visual branches while retaining all action supervision, encouraging use of shared observations.
Abstract (from arXiv)
Multi-robot control requires integrating distributed camera views while predicting separate action sequences. Most vision-language-action and world-action models target one robot. Combining multiple views alone does not establish spatial correspondence between them. World Dreamer v5-lite is a multi-robot world-action model built around a shared world state for robot-specific action prediction. Calibrated features from a pretrained video encoder populate this state in workspace coordinates; its spatial size remains fixed as views are added. A vision-language model provides scene context and one query per robot. The queries read the shared state and exchange pose-aware information, while their action streams remain separate. Masked-view cooperative training withholds selected camera views from both visual branches while retaining all action targets. This encourages the use of observations shared by other robots. Pretraining uses more than 10,000 hours of trajectories from 20 simulated robot types, RoboFactory, and real robots. On RoboFactory, World Dreamer v5-lite achieves 74.0% mean success, 23.7 percentage points above the strongest evaluated baseline. Across four real-robot tasks, it reaches 78.8% mean success, a margin of 13.8 points. It also reaches 58.08% overall on RoboCasa365.
Related papers
- DreamTrue: Action-Faithful Robot World Model with Counterfactual Post-Training
- UniWAM: Unified World-Action Model
- Learning Robotic World Models with Semantic and Visual Feedback
- SUAVE: Unified Video-Action Models via Masked Diffusion
- RoboSeed: Multidimensional Evaluation of Vision-Language-Action and World-Action Models: Capability, Resource Affordability, and Research Evidence
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-27 · Indexed by Pub-AI: 2026-10-28 UTC
World Dreamer v5-lite: A Multi-Robot World Action Model with One Dreamer
Preprint posted: 2026-10-27 · Original record ↗
- arxiv
- 2610.37518
Explicit source versions (1)
- v1 · 2026-10-27
- Preprint available
Source date: 2026-10-27 · First detected: 2026-10-28 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.37518v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Multi-robot control requires integrating distributed camera views while predicting separate action sequences. Most vision-language-action and world-action models target one robot. Combining multiple views alone does not establish spatial correspondence between them. World Dreamer v5-lite is a multi-robot world-action model built around a shared world state for robot-specific action prediction. Calibrated features from a pretrained video encoder populate this state in workspace coordinates; its spatial size remains fixed as views are added. A vision-language model provides scene context and one query per robot. The queries read the shared state and exchange pose-aware information, while their action streams remain separate. Masked-view cooperative training withholds selected camera views from both visual branches while retaining all action targets. This encourages the use of observations shared by other robots. Pretraining uses more than 10,000 hours of trajectories from 20 simulated robot types, RoboFactory, and real robots. On RoboFactory, World Dreamer v5-lite achieves 74.0% mean success, 23.7 percentage points above the strongest evaluated baseline. Across four real-robot tasks, it reaches 78.8% mean success, a margin of 13.8 points. It also reaches 58.08% overall on RoboCasa365.
