Latent Dynamics · 2026-10-01
Why Do Conventional World Models Fail to Learn Cellular Automata?
Shaoyang Guo, Ziming Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors test conventional world models (CNNs, transformers, diffusion) on cellular automata and find they often predict many pixels correctly but rarely complete exact long-horizon rollouts. They attribute failure to three inductive-bias gaps: spatial locality, temporal locality, and temporal stability. Simple information-flow changes (2D rotary positions, momentum induction, causal freezing) lift rollout success dramatically on Game of Life and unseen rules.
Why it matters
The paper argues that good short-horizon pixel accuracy can hide rule-level failures: conventional autoregressive or diffusion models may not reproduce exact dynamics over rollouts. By using cellular automata as a controlled testbed, the authors isolate three failure modes (spatial locality, temporal locality, temporal stability) and show that each can be repaired without changing the architectural backbone, suggesting concrete design principles for world models meant for long-horizon generation/control.
Method
- Evaluate CNN, transformer, and diffusion next-frame models on binary outer-totalistic cellular automata on an 8×8 torus; train on 8 observed frames and score long-horizon rollout success with SeqAcc (all 8×64 cells correct).
- Diagnose three failure modes: missing spatial locality (flattened raster positions), missing temporal locality (no stable pairing of a situation with its successor frame), and missing temporal stability (joint denoising revise frames).
- Repair each failure with backbone-preserving information-flow changes: 2D rotary positions; momentum induction (store situation with its continuation via previous-frame 3×3×3 neighbourhood); causal freezing (denoise frames in order and freeze as context).
Abstract (from arXiv)
Although conventional world models - auto-regressive or diffusion models based on transformers or convolutional networks - may learn surface statistics of world dynamics, can they learn the exact world dynamics from its observed history? Leveraging cellular automata as a simple testbed, we find the answer to be no in many cases. Conventional architectures predict most pixels correctly yet rarely complete a rollout: a CNN predicts 96.3% of cells but completes 18.9% of rollouts; a joint diffusion model completes none. We trace the gap to three failure modes of these world models - namely, they fail to exactly capture spatial locality, temporal locality or temporal stability. Simple changes repair each: (1) for spatial locality, two-dimensional rotary positions lift a transformer from 39.1% to 100% on the Game of Life; (2) for temporal locality, handing each token its cell's previous-frame neighbourhood lifts the same transformer from 25.8% to 99.9% on unseen rules; (3) for temporal stability, causal freezing lifts the same diffusion weights from 42.2% to 99.9%. None of the three changes touches the architectural backbone; each only modifies the information flow within it. We also compare joint and ordered sampling on billiards and, in an exploratory study, on a simulated Burgers equation.
Related papers
- WorldFoundry: Towards Unified Infrastructure for World Intelligence
- Rethinking Video Autoencoders for Interactive World Models
- world2vec: Mechanistic world models through latent dynamics learning
- Reading, Writing, Sharing, and Interacting with the World through Video
- WorldCast: Exploring Intelligent Visual Generation in World State Prediction
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-30 · Indexed by Pub-AI: 2026-10-01 UTC
Why Do Conventional World Models Fail to Learn Cellular Automata?
Preprint posted: 2026-09-30 · Original record ↗
- arxiv
- 2609.39604
Explicit source versions (1)
- v1 · 2026-09-30
- Preprint available
Source date: 2026-09-30 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Although conventional world models - auto-regressive or diffusion models based on transformers or convolutional networks - may learn surface statistics of world dynamics, can they learn the exact world dynamics from its observed history? Leveraging cellular automata as a simple testbed, we find the answer to be no in many cases. Conventional architectures predict most pixels correctly yet rarely complete a rollout: a CNN predicts 96.3% of cells but completes 18.9% of rollouts; a joint diffusion model completes none. We trace the gap to three failure modes of these world models - namely, they fail to exactly capture spatial locality, temporal locality or temporal stability. Simple changes repair each: (1) for spatial locality, two-dimensional rotary positions lift a transformer from 39.1% to 100% on the Game of Life; (2) for temporal locality, handing each token its cell's previous-frame neighbourhood lifts the same transformer from 25.8% to 99.9% on unseen rules; (3) for temporal stability, causal freezing lifts the same diffusion weights from 42.2% to 99.9%. None of the three changes touches the architectural backbone; each only modifies the information flow within it. We also compare joint and ordered sampling on billiards and, in an exploratory study, on a simulated Burgers equation.
