Video World Models · 2026-10-22
How Representational Priors Shape Generalization in World Action Models
Junha Chun, Youngjoon Jeong, Taesup Kim
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study world action models (WAMs) and find that representational geometry inherited from generative backbones predicts generalization under visual shifts. They define nuisance orthogonality (feature displacement orthogonal to the empirical in-distribution manifold) and show higher orthogonality is associated with stronger LIBERO-Plus policy success after WAM training. After training, they analyze nuisance suppression (reduced residual displacement) as an explanation, with ImageWAM showing stronger suppression than FastWAM.
Why it matters
Understanding why WAMs generalize under distribution shift is important for selecting backbone designs. The paper proposes nuisance orthogonality as a prospective diagnostic before policy training, and nuisance suppression as a representation-level mechanism after adaptation. The reported LIBERO-Plus results relate backbone geometry to downstream visual robustness, and the analysis provides geometric evidence for how robust policies suppress nuisance information.
Method
- Define nuisance orthogonality as orthogonality of representation displacement under visual transformations relative to the empirical in-distribution feature span, aggregated across transformer layers and a diagnostic dataset from LIBERO-Plus.
- Compare generative backbones (e.g., Wan2.2, FLUX.2, OmniGen2, Ovis-U1, plus extended backbones) by measuring nuisance orthogonality before WAM training and relating it to policy success rates after WAM training on LIBERO-Plus.
- Analyze nuisance suppression after WAM adaptation using normalized standardized in-distribution and orthogonal response components (ratios ρ_T and ρ⊥,T) to quantify relative suppression of residual displacement.
Limitation
The authors state their evidence is restricted to simulated LIBERO manipulation and visual shifts evaluated in LIBERO-Plus, excluding generalization benchmarks for language instructions or action compositionalities; results are observational and do not isolate every causal effect of orthogonality or of pretraining objective.
Abstract (from arXiv)
World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical in-distribution manifold. Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models. In addition, analyzing the resulting robust WAMs reveals nuisance suppression; relative suppression of feature displacement caused by such variations and robust action generation of the policy. Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.
Related papers
- What Visual Representations Do World Action Models Need?
- ACG-WAM: World-Action Modeling via Action-Conditioned Geometric Latent Prediction
- Counterfactual Action Evaluation, Observation Bottlenecks, and Representation Geometry in Joint-Embedding Predictive World Models
- Rethinking Representations for World-Action Modeling
- {\Delta}WAM: Distilling Action Tangent Fields into World Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-21 · Indexed by Pub-AI: 2026-10-22 UTC
How Representational Priors Shape Generalization in World Action Models
Preprint posted: 2026-10-21 · Original record ↗
- arxiv
- 2610.29168
Explicit source versions (1)
- v1 · 2026-10-21
- Preprint available
Source date: 2026-10-21 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World action models (WAMs) are shown to substantially diverge under generalization performance based on their generative backbones. We investigate this gap through the representational geometry that WAMs inherit from their backbones and how policy training reorganizes it. Our central hypothesis is that a useful representational prior would structure geometry that separates nuisance information from the in-distribution manifold in advance of training as a policy. We quantify this with nuisance orthogonality; orthogonality of the feature under visual variation outside the empirical in-distribution manifold. Across various image and video backbones, we show that a higher orthogonality from the representational prior is associated with stronger robustness to visual variations after WAM training, enabling policy generalization measurable prospectively for world models. In addition, analyzing the resulting robust WAMs reveals nuisance suppression; relative suppression of feature displacement caused by such variations and robust action generation of the policy. Together, these findings provide a geometric perspective on how the representational prior of the world model shapes the generalization performance of WAMs.