Events, Not Pixels: Where World Models Get the Return Wrong
The authors analyze why pixel world models can mis-rank actions during planning: return error can concentrate at “event transitions” (reward earned or episode ends) even when those transitions look visually normal. They introduce a transition-level substitution diagnostic that swaps imagined transitions with real ones under budget-matched random controls, and show event patches remove much more return error than patches chosen by pixel metrics.
