Robotics · 2026-10-08
World Models Dream of Success: Diagnosing and Repairing Failure Insensitivity in Robot World Models
Jiuyi Xu, Xiao Hu, Meida Chen, Peng Gao, Yang Ye, Yangming Shi
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Robot world models can be failure-insensitive, producing success-like predictions on verified failures. The authors introduce CureWM: construct execution-verified counterfactual failures by perturbing successful demonstrations across a severity grid, then fine-tune released models on verified failing and surviving successful replays without changing architecture or training objective. On 484 held-out LIBERO failure counterfactuals, optimism drops to 30–43% (38% mean); on a physical Franka arm, failure diagnostic optimism drops from 90% to 33%.
Why it matters
For planning and policy evaluation, world models must distinguish successful actions from failures. The paper shows that reduced optimism alone can overstate repair, and that improving discrimination needs counterfactual supervision from shared contexts with execution-verified outcomes. CureWM targets post-hoc repair of released robot world models under fixed architecture and objectives, and the evaluations include both simulation and physical-robot checks.
Method
- Diagnose failure insensitivity using success–failure value gaps and success-like prediction diagnostics (e.g., optimism-rate, AUROC).
- Construct counterfactual action alternatives by perturbing successful demonstrations across a severity grid (while preserving pre-intervention actions), then verify outcomes by execution in simulation or on hardware.
- Fine-tune released models on their original objective using the base training data plus converted verified counterfactual replays (failures and surviving successes), with architecture and training objective unchanged.
Limitation
Their video-based diagnostics require recorded reference outcomes; the paper notes that a reduction in mean score is not by itself proof of more accurate generated video.
Abstract (from arXiv)
Robot world models support policy evaluation, planning, and synthetic data generation, but these applications require predictions that distinguish successful actions from failures. Across four released checkpoints from two architecture families, we observe weak sensitivity to action changes and success-like predictions on verified failures. Although recent work incorporates failures into model training, which data can repair released checkpoints without changing their architecture or training objective still remains underexplored. To this end, we introduce CureWM, which constructs alternative actions from successful demonstrations across a severity grid, verifies their outcomes through execution in simulation or on hardware, and fine-tunes released models on the resulting failures and surviving successes alongside nominal demonstrations. This construction provides controlled action contrasts from shared starting contexts. On 484 held-out LIBERO failure counterfactuals, optimism falls from 80\% after fine-tuning on the official data to 30--43\% across four independently fine-tuned CureWM models (38\% mean). In two separate evaluations on a physical robot arm, failure predictions scored as success-like by a latent-distance diagnostic decrease from 90\% after fine-tuning on successful demonstrations alone to 33\% with CureWM. With failure counts per task, successful replay data, and training budget matched, counterfactual failures yield a success--failure value gap of 0.124, compared with 0.014 for freshly collected on-policy failures. These findings support execution-verified counterfactual replay for post-hoc repair and show why reduced optimism must be evaluated alongside success--failure discrimination. Code is available at https://github.com/jiuyixu25/CureWM.
Related papers
- $R^2$-WAM: Repair-and-Reject Post-Training for World Action Models
- Events, Not Pixels: Where World Models Get the Return Wrong
- RoboCoach: World Models as Active Coaches for Compositional Robot Skills
- Future Anchored Verification and Online Recovery for World Action Models
- When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-08 UTC
World Models Dream of Success: Diagnosing and Repairing Failure Insensitivity in Robot World Models
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.09134
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Robot world models support policy evaluation, planning, and synthetic data generation, but these applications require predictions that distinguish successful actions from failures. Across four released checkpoints from two architecture families, we observe weak sensitivity to action changes and success-like predictions on verified failures. Although recent work incorporates failures into model training, which data can repair released checkpoints without changing their architecture or training objective still remains underexplored. To this end, we introduce CureWM, which constructs alternative actions from successful demonstrations across a severity grid, verifies their outcomes through execution in simulation or on hardware, and fine-tunes released models on the resulting failures and surviving successes alongside nominal demonstrations. This construction provides controlled action contrasts from shared starting contexts. On 484 held-out LIBERO failure counterfactuals, optimism falls from 80\% after fine-tuning on the official data to 30--43\% across four independently fine-tuned CureWM models (38\% mean). In two separate evaluations on a physical robot arm, failure predictions scored as success-like by a latent-distance diagnostic decrease from 90\% after fine-tuning on successful demonstrations alone to 33\% with CureWM. With failure counts per task, successful replay data, and training budget matched, counterfactual failures yield a success--failure value gap of 0.124, compared with 0.014 for freshly collected on-policy failures. These findings support execution-verified counterfactual replay for post-hoc repair and show why reduced optimism must be evaluated alongside success--failure discrimination. Code is available at https://github.com/jiuyixu25/CureWM.