Driving · 2026-10-14
WoVR+: World Model as Reliable Simulator for Versatile VLA Policy Optimization with RL
Shangqing Zhou, Zhennan Jiang, Ning Ma, Xin Fu, Yupeng Zheng, Shuai Tian, Huangrui Li, Jiahao Liu, Chao Yu, Zhongpu Xia, Haoran Li, Dongbin Zhao
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WoVR+ extends WoVR’s hallucination-aware WM-RL to post-train Vision–Language–Action (VLA) policies with richer multi-view inputs and long-horizon, multi-stage tasks. It uses a VLA-interface-aligned multi-view world model with geometry-grounded action/camera conditioning and command-based proprioceptive updates, plus chunk-wise Flow-SDE exploration. For long horizons, it uses stage-local imagined training with stage-entry resets, stage-specific rewards, local horizons, and controllable sampling, improving success over SFT on RoboTwin and 6 real-robot tasks.
Why it matters
Action-conditioned world models can hallucinate in closed-loop rollouts, potentially producing spurious success signals that mislead RL. The authors aim to make WM-RL reliable for modern VLA interfaces (multi-view + proprioception) and long-horizon manipulation by explicitly redesigning the learned simulator/interface and the imagined interaction/training protocol.
Method
- VLA-interface-aligned multi-view world model: geometry-grounded action/camera conditioning with view-specific action maps and raymaps, anchored sparse history, and command-based proprioceptive updates under an absolute-joint-and-gripper interface.
- Temporally coherent exploration: chunk-wise Flow-SDE exploration for absolute-action chunks to preserve meaningful exploration while suppressing exploration-induced action jitter.
- Stage-local imagined optimization for long horizons: stage-entry resets, stage-specific rewards, local rollout horizons, and controllable stage sampling to shorten generated prefixes and balance learning across stages.
Abstract (from arXiv)
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors. We present WoVR+, extending WoVR's hallucination-aware WM-RL framework by redesigning the learned environment and imagined interaction protocol: A VLA-interface-aligned world model provides multi-view prediction and proprioceptive updates while mitigating hallucinations. Chunk-wise noise preserves flow-based policy exploration while limiting prediction-degrading action jitter. For long-horizon tasks, stage-local imagined training combines stage-entry resets, stage-specific rewards, local horizons, and controllable sampling to shorten generated prefixes and balance training across stages. WoVR+ improves success over SFT by 21.09% and 16.7% on RoboTwin and 6 real-robot tasks. Long-horizon phone-packaging success rises from 30% to 75%. Videos are available at https://youmo445.github.io/WoVR_Plus.github.io/.
Related papers
- DRILL: Training World Models to Improve Policies, Not Predict Pixels
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- PLaW-VLA: Predictive Latent World Modeling for Vision-Language-Action Policies
- The Planning Limits of Latent World Models
- DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
WoVR+: World Model as Reliable Simulator for Versatile VLA Policy Optimization with RL
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.17922
Explicit source versions (1)
- v1 · 2026-10-13
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Action-conditioned world models can reduce costly real-robot interaction in reinforcement learning for Vision-Language-Action models (VLA-RL), but hallucinations can mislead policy optimization. Extending them to richer VLA interfaces and long-horizon tasks raises challenges in spatiotemporal and cross-modal consistency, exploration-reliability balance in RL, and compounding rollout errors. We present WoVR+, extending WoVR's hallucination-aware WM-RL framework by redesigning the learned environment and imagined interaction protocol: A VLA-interface-aligned world model provides multi-view prediction and proprioceptive updates while mitigating hallucinations. Chunk-wise noise preserves flow-based policy exploration while limiting prediction-degrading action jitter. For long-horizon tasks, stage-local imagined training combines stage-entry resets, stage-specific rewards, local horizons, and controllable sampling to shorten generated prefixes and balance training across stages. WoVR+ improves success over SFT by 21.09% and 16.7% on RoboTwin and 6 real-robot tasks. Long-horizon phone-packaging success rises from 30% to 75%. Videos are available at https://youmo445.github.io/WoVR_Plus.github.io/.