Driving · 2026-10-14
ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation
Zijun Wang, Tianyi Yan, Haokun Lin, Panwen Hu, Bohang He, Siyu Ren, Xiaoyu Guo, Weijia Liufu, Shuoshuo Xue, Fengjiao Chen, Zipeng Ye, Xuezhi Cao, Zutao Jiang, Hanhui Li, Xiaodan Liang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ProWorld is an action-conditioned, multi-view robot world model for closed-loop policy evaluation. It jointly predicts multi-view video and proprioceptive state, feeds predicted observations back to a frozen policy, and includes failure-aware training using failed interactions from high-fidelity simulation to reduce interaction hallucinations. The authors report ProWorld achieves the highest overall TriWorldBench score and strong correlation with real success rates.
Why it matters
The paper targets reliable evaluation of robot manipulation policies in generated closed-loop environments. It argues evaluators need complete visual–proprioceptive feedback and faithful failure modeling to prevent success-rate inflation from hallucinated outcomes, and it proposes components (joint video-state prediction, STVE, and failure-aware training) aimed at those requirements.
Method
- ProWorld treats rollout as a repeated observe–act loop: initialized from a real observation, it predicts multi-view observations and robot states for H steps, then feeds both back to a frozen policy for closed-loop interaction.
- The model uses a shared transformer for joint video–state flow prediction with asymmetric attention: state queries read visual dynamics, while motion/state losses supervise the shared backbone.
- To preserve camera identity and timing, it introduces Shared-Time View Encoding (STVE); to reduce interaction hallucinations, it augments training with failed interactions collected in high-fidelity simulation.
Limitation
Object deformation and drift over exceptionally long rollouts remain limitations. The authors state that improving geometry persistence and rollout stability could reduce accumulated evaluation errors.
Abstract (from arXiv)
Evaluating robot manipulation policies through repeated physical trials is costly. We study whether an action-conditioned world model can serve as a closed-loop test environment whose policy success rates track those on real robots. Beyond realistic video, the model must return the camera views and proprioceptive state needed for the policy's next action and preserve failed-action outcomes to avoid overestimating policy capability. We introduce ProWorld, a world model that jointly predicts multi-view video and robot state, then feeds both back to a frozen policy for repeated interaction. ProWorld comprises a shared transformer for joint video-state flow prediction and Shared-Time View Encoding to distinguish cameras while preserving temporal synchronization. To reduce interaction hallucinations that turn missed grasps into successful lifts, we augment training with failed interactions collected in high-fidelity simulation, reducing inflated success rates in the generated environment. ProWorld achieves the highest overall TriWorldBench score among the compared methods. Across three policies and four local manipulation tasks, it attains a Pearson correlation of 0.95 and a Spearman correlation of 0.94 between generated and real success rates. These results support joint visual-proprioceptive feedback and failure-aware training as a basis for world-model-based policy evaluation.
Related papers
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
- Physical Canvas: A Vision Tactile World Model For Robot Policy Evaluation And Learning
- Beyond Successful Robot Demonstrations: Learning from Human Interactions and Modeling Failure Dynamics
- DeltaWorld: Physically Consistent Interactive World Simulators via Action-Conditioned Latent Increment Learning
- Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-11 · Indexed by Pub-AI: 2026-10-14 UTC
ProWorld: Multi-View World Model with Proprioceptive Feedback for Closed-Loop Policy Evaluation
Preprint posted: 2026-10-11 · Original record ↗
- arxiv
- 2610.15294
Explicit source versions (1)
- v1 · 2026-10-11
- Preprint available
Source date: 2026-10-11 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Evaluating robot manipulation policies through repeated physical trials is costly. We study whether an action-conditioned world model can serve as a closed-loop test environment whose policy success rates track those on real robots. Beyond realistic video, the model must return the camera views and proprioceptive state needed for the policy's next action and preserve failed-action outcomes to avoid overestimating policy capability. We introduce ProWorld, a world model that jointly predicts multi-view video and robot state, then feeds both back to a frozen policy for repeated interaction. ProWorld comprises a shared transformer for joint video-state flow prediction and Shared-Time View Encoding to distinguish cameras while preserving temporal synchronization. To reduce interaction hallucinations that turn missed grasps into successful lifts, we augment training with failed interactions collected in high-fidelity simulation, reducing inflated success rates in the generated environment. ProWorld achieves the highest overall TriWorldBench score among the compared methods. Across three policies and four local manipulation tasks, it attains a Pearson correlation of 0.95 and a Spearman correlation of 0.94 between generated and real success rates. These results support joint visual-proprioceptive feedback and failure-aware training as a basis for world-model-based policy evaluation.