Video World Models · 2026-10-21
Training Controllable World Models via Action Influence Maximization
Samuel M. Bateman, Tenny Yin, Chongyi Zheng, Lei Huang, Brian Wang, Benjamin Eysenbach, Jaime Fern\'andez Fisac, Dhruv Shah
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Action-conditioned video world models trained on successful demonstrations can ignore commanded actions due to insufficient action influence in the dataset. The authors analyze an explained-action-information metric and introduce Action Influence Maximization (AIM) to actively collect data that increases dataset mutual information about actions and next states. In pixel manipulation (Push-T), AIM is reported as the only method that consistently improves controllability, matching a 300k expert model with only 500 actively collected episodes added to 3k demonstrations.
Why it matters
The paper targets a controllability failure mode of action-conditioned world models: expert-like futures that are insensitive to commanded actions. By linking controllability to dataset conditional mutual information and proving an upper bound from dataset properties, the authors justify active data collection as a way to obtain the missing action consequences needed for downstream policy evaluation, especially where passive data scaling saturates.
Method
- Analyze controllability via explained action information (prediction error difference with and without knowing the action) and relate it to conditional mutual information from the dataset.
- Prove that dataset conditional mutual information upper-bounds explained action information that any model trained on that data can express.
- Introduce AIM: an active data collection algorithm that uses a learned intrinsic reward combining reference state coverage and a pointwise mutual information term, optimized in a batch online RL loop with conservative offline RL.
Limitation
In Section 11.3/13, the authors hypothesize that tee-related simulation fidelity remains difficult due to poor coverage of tee positions in the dataset and lack of broad pre-training data, and they also note extrapolation to more off-policy “push away” states remains difficult without notably more active data collection or additional on-policy expert demonstrations.
Abstract (from arXiv)
Action-conditioned video generative models trained on successful, goal-directed demonstrations tend to ignore the commanded action, rendering a plausible expert-like future regardless of what is asked of them. We trace this failure to training data that lack sufficient action influence. We analyze this metric and prove that it upper bounds the action influence that any model trained on that data can express. Expert demonstrations, whose narrow action distributions let next states be predicted without attending to the action, set a low ceiling that no amount of model capacity alone can raise. Raising it requires collecting different data. We introduce Action Influence Maximization (AIM), an algorithm for actively collecting training data for improving dataset action influence. We compare the performance and data-efficiency of AIM against exploration heuristics and intrinsic-motivation methods on simulated navigation and manipulation tasks. In pixel-based manipulation, AIM is the only method that consistently improves controllability: 500 actively collected episodes added to 3k demonstrations match a world model trained on 300k expert episodes on end-effector controllability, an $85\times$ reduction in data.
Related papers
- Direct Experience World-Model Optimization: Learning the World Beyond Action Imitation
- Video Prediction Policy 2: Predict Better, Act Better
- W2A: Distilling World Priors into Action Models for Real-Time Robotic Control
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
- What Limits Cross-Task Generalization in World Action Models?
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-21 UTC
Training Controllable World Models via Action Influence Maximization
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.27387
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (3)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Action-conditioned video generative models trained on successful, goal-directed demonstrations tend to ignore the commanded action, rendering a plausible expert-like future regardless of what is asked of them. We trace this failure to training data that lack sufficient action influence. We analyze this metric and prove that it upper bounds the action influence that any model trained on that data can express. Expert demonstrations, whose narrow action distributions let next states be predicted without attending to the action, set a low ceiling that no amount of model capacity alone can raise. Raising it requires collecting different data. We introduce Action Influence Maximization (AIM), an algorithm for actively collecting training data for improving dataset action influence. We compare the performance and data-efficiency of AIM against exploration heuristics and intrinsic-motivation methods on simulated navigation and manipulation tasks. In pixel-based manipulation, AIM is the only method that consistently improves controllability: 500 actively collected episodes added to 3k demonstrations match a world model trained on 300k expert episodes on end-effector controllability, an $85\times$ reduction in data.
