Video World Models · 2026-10-07

Identifiable World Models from Pretrained Diffusion Representations

Ruchi Sandilya, Conor Liston, Logan Grosenick

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors study whether a frozen pretrained diffusion model for video can be equipped with identifiable latent coordinates without retraining its generative backbone. They propose Contrastive Diffusion Alignment (ConDA), learning only a lightweight alignment map on frozen diffusion latents. Under TCL/GCL-style assumptions, the aligned coordinates identify latent dynamical states (up to permutation and componentwise invertible transforms), preserve the latent dynamic SCM, and enable lagged causal graph recovery via inverse transition-noise Jacobian sparsity.

Why it matters

Predictive world-model latents may not correspond to underlying state variables or causal mechanisms needed for intervention and control. This work separates diffusion generative fidelity from causal identification: keep the diffusion backbone frozen for generation, and learn an auxiliary-aligned coordinate chart whose variables are identifiable and structurally interpretable, enabling causal graph recovery and analysis of intervention-relevant dynamics.

Method

Train a conditional latent diffusion model on video, then freeze it. Invert each frame into frozen diffusion latents x_t (via inversion steps) and learn only an alignment map z_t = h(x_t) using TCL, GCL, or CEBRA objectives (ConDA).
The authors connect auxiliary-aligned contrastive learning on diffusion latents to auxiliary-variable nonlinear ICA guarantees, yielding identifiability of latent dynamical states up to permutation and componentwise invertible transformations under TCL/GCL assumptions.
For causal discovery, they show lagged causal edges can be characterized by sparsity of the inverse transition-noise Jacobian; empirically they estimate graphs from learned forward-dynamics Jacobians (or controlled transition models for Microduck).

Abstract (from arXiv)

Diffusion-based world models can generate and predict trajectories in high-dimensional dynamical systems, but predictive accuracy does not imply that their latent coordinates recover the underlying state variables or causal interactions. We ask whether a frozen pretrained diffusion model can be equipped with identifiable coordinates without retraining its generative backbone. We show that auxiliary-variable nonlinear ICA guarantees can be transferred to Contrastive Diffusion Alignment (ConDA), which learns only a lightweight alignment map on top of frozen diffusion latents. Under standard TCL/GCL assumptions, the aligned representation identifies latent dynamical states up to permutation and componentwise invertible transformations, preserves the latent dynamic structural causal model, and reduces lagged graph recovery to transition-Jacobian sparsity. We evaluate TCL-, GCL-, and CEBRA-based ConDA against TDRL, CaRiNG, IDOL, temporal SuaVE, and iVAE across physical and robotic video systems. TCL and GCL achieve near-perfect blockwise state recovery and competitive lagged graph recovery, including exact recovery in a simulated falling-body system. In a simulated bipedal robot, learned dynamics recover the sign and temporal structure of responses to held-out control perturbations. These results show that a frozen generative diffusion model can be equipped with coordinates that are identifiable, structurally interpretable, and useful for analyzing intervention-relevant dynamics.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-07 UTC

preprint