Latent Dynamics · 2026-10-22

Events, Not Pixels: Where World Models Get the Return Wrong

Haoyue Liu, Xiaoying Tang

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors analyze why pixel world models can mis-rank actions during planning: return error can concentrate at “event transitions” (reward earned or episode ends) even when those transitions look visually normal. They introduce a transition-level substitution diagnostic that swaps imagined transitions with real ones under budget-matched random controls, and show event patches remove much more return error than patches chosen by pixel metrics.

Why it matters

Planning needs correct return estimates at decision-relevant transitions, not just visually faithful rollouts. The authors’ diagnostic localizes where return error originates within imagined rollouts, showing that event transitions drive planning mistakes and that similar concentration appears on other control tasks with rare reward events. This motivates evaluating and repairing world models for planning based on return-error localization rather than frame rendering quality alone.

Method

Introduce a transition-level substitution diagnostic: pair each imagined rollout with a real rollout sharing context and recorded actions; substitute real vs imagined transition frames (or reward/termination signals) at selected time steps using a frozen, memoryless detector to measure return changes.
Use budget-matched controls (same number of patched steps) including patches at true events, random steps, non-event steps, and an all-frame real reference; measure fractional reduction in mean absolute return error and targeting lift.
Test event absence vs mistiming via reward-event recall under alignment tolerance, and test repair via replacing a deterministic MSE predictor with a diffusion model sampled on the same data; evaluate event recall, return error, and plan ranking.

Limitation

The diagnostic requires only rendered frames, but it depends on a frozen detector trained once per game; games whose detector fails the gate are omitted (supplement).

Abstract (from arXiv)

World models are increasingly used for planning, where a planner ranks candidate actions by returns imagined by the model. However, these models are judged mainly by how well they render frames, even though a visually faithful rollout can still assign the wrong return. To identify where this mismatch arises, we introduce a transition-level diagnostic that replaces imagined transitions with real ones and measures the reduction in return error against a budget-matched random control. Our analysis yields two findings. (1) Event transitions, the few steps where a reward is earned or an episode ends, are rendered as well as other steps (median pixel-error ratio 0.85), yet restoring them removes a median of 70% of the return error. In contrast, restoring transitions with the largest MSE, SSIM or LPIPS error removes less than a random choice. Across the 44 Atari-100k model--collection pairs that pass our reliability checks, including three released families, event patches outperform random patches in 39 (sign test p=1.4 x 10^{-7}), with a median gain of +44 percentage points. (2) The remaining error is structured: a complementary patch separates errors at true events from those elsewhere, while an eight-step timing tolerance recovers few events missed by our deterministic models. This structure also guides repair: a rule recorded before four repair outcomes were known predicts when a diffusion model repairs these failures. Event patches improve plan ranking, and the same concentration appears on control tasks with rare reward events.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-22 UTC

preprint