Video World Models · 2026-10-14

World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models

Shinseong Kim, Jaeseok Jeong, Jianlong Fu, Youngjung Uh

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Physical Registry introduces temporally grounded registry slots inside a video DiT to hold predictive future dynamics. The registry is initialized from an observation-conditioned JEPA forecast, co-evolves with video tokens, and receives future-representation supervision, making predictive dynamics an explicit readable model state. It improves Physics-IQ Verified and WorldModelBench physical adherence, and supports Physical Registry Selector (PRS) for candidate selection during denoising.

Why it matters

The paper argues that video world models need an internal, readable state for scene evolution rather than relying on entangled tokens or external physical guidance. By making predictive dynamics an explicit state inside the generator and supervising it toward future representations, Physical Registry improves physical plausibility metrics and enables intermediate candidate ranking (PRS) without completed-video external scoring.

Method

Augment flow-matching video DiTs with Physical Registry: forecast-initialized registry slots (spatial grid per future temporal position) concatenated with noisy video latents and jointly processed by the DiT.
Initialize registry at each denoising step from a JEPA encoder–predictor forecast from the observed frame; supervise intermediate registry readouts toward target future representations.
Use PRS to rank candidate trajectories by how much their intermediate registry readouts depart from the shared forecast, selecting a candidate before fully denoising.

Limitation

The paper does not state any explicit limitation in the provided text.

Abstract (from arXiv)

Video world models must generate physically plausible futures, not just realistic frames. Existing approaches typically use semantic, physical, or predictive information to guide the visual generation stream, leaving a fundamental question: where inside the generator should predictions of scene evolution live? Standard video tokens jointly encode appearance and dynamics, but provide no distinct state for learning and reading predictive dynamics. We answer this question with the Physical Registry: temporally grounded registry slots inside a video DiT that are initialized from an observation-conditioned forecast, co-evolve with video tokens, and receive future-representation supervision, making predictive dynamics an explicit, readable state of the generator. In a controlled comparison of predictive-state interfaces on Physics-IQ Verified, Physical Registry improves vanilla fine-tuning by 9.9/8.7 points on I2V/V2V, the largest gain among the interfaces we compare. Across all Cosmos and WAN backbones, Physical Registry improves Physics-IQ Verified scores. With only 0.45% additional parameters, it enables Cosmos 3 Nano 16B to surpass the published I2V score of Cosmos 3 Super 64B (43.56 vs. 42.7). On WorldModelBench, it also improves physical adherence and the total score for all four evaluated backbones. Because the registry becomes candidate-specific while remaining readable, the Physical Registry Selector (PRS) ranks emerging futures by how far their intermediate registry readouts depart from the shared forecast before full denoising is complete. Across four backbones, this selection improves I2V scores by 5.6-15.9% without completed-video external scoring. Together, these results support a broader principle: dynamics prediction should live as an explicit model state that co-evolves with the video representation. Project page: https://grainsack.github.io/physical-registry/

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-12 · Indexed by Pub-AI: 2026-10-14 UTC

preprint