Video World Models · 2026-10-21

Training Controllable World Models via Action Influence Maximization

Samuel M. Bateman, Tenny Yin, Chongyi Zheng, Lei Huang, Brian Wang, Benjamin Eysenbach, Jaime Fern\'andez Fisac, Dhruv Shah

arXiv:2610.27387PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Action-conditioned video world models trained on successful demonstrations can ignore commanded actions due to insufficient action influence in the dataset. The authors analyze an explained-action-information metric and introduce Action Influence Maximization (AIM) to actively collect data that increases dataset mutual information about actions and next states. In pixel manipulation (Push-T), AIM is reported as the only method that consistently improves controllability, matching a 300k expert model with only 500 actively collected episodes added to 3k demonstrations.

Why it matters

The paper targets a controllability failure mode of action-conditioned world models: expert-like futures that are insensitive to commanded actions. By linking controllability to dataset conditional mutual information and proving an upper bound from dataset properties, the authors justify active data collection as a way to obtain the missing action consequences needed for downstream policy evaluation, especially where passive data scaling saturates.

Method

  • Analyze controllability via explained action information (prediction error difference with and without knowing the action) and relate it to conditional mutual information from the dataset.
  • Prove that dataset conditional mutual information upper-bounds explained action information that any model trained on that data can express.
  • Introduce AIM: an active data collection algorithm that uses a learned intrinsic reward combining reference state coverage and a pointwise mutual information term, optimized in a batch online RL loop with conservative offline RL.

Limitation

In Section 11.3/13, the authors hypothesize that tee-related simulation fidelity remains difficult due to poor coverage of tee positions in the dataset and lack of broad pre-training data, and they also note extrapolation to more off-policy “push away” states remains difficult without notably more active data collection or additional on-policy expert demonstrations.

Abstract (from arXiv)

Action-conditioned video generative models trained on successful, goal-directed demonstrations tend to ignore the commanded action, rendering a plausible expert-like future regardless of what is asked of them. We trace this failure to training data that lack sufficient action influence. We analyze this metric and prove that it upper bounds the action influence that any model trained on that data can express. Expert demonstrations, whose narrow action distributions let next states be predicted without attending to the action, set a low ceiling that no amount of model capacity alone can raise. Raising it requires collecting different data. We introduce Action Influence Maximization (AIM), an algorithm for actively collecting training data for improving dataset action influence. We compare the performance and data-efficiency of AIM against exploration heuristics and intrinsic-motivation methods on simulated navigation and manipulation tasks. In pixel-based manipulation, AIM is the only method that consistently improves controllability: 500 actively collected episodes added to 3k demonstrations match a world model trained on 300k expert episodes on end-effector controllability, an $85\times$ reduction in data.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-21 UTC

preprint

Training Controllable World Models via Action Influence Maximization

Samuel M. Bateman, Tenny Yin, Chongyi Zheng, Lei Huang, Brian Wang, Benjamin Eysenbach, Jaime Fernández Fisac, Dhruv Shah

Preprint posted: 2026-10-19 · Original record ↗

arxiv
2610.27387
Explicit source versions (1)
  • v1 · 2026-10-19
  1. Preprint available

    Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline

    Source evidence (3)
Abstract · arxiv

Action-conditioned video generative models trained on successful, goal-directed demonstrations tend to ignore the commanded action, rendering a plausible expert-like future regardless of what is asked of them. We trace this failure to training data that lack sufficient action influence. We analyze this metric and prove that it upper bounds the action influence that any model trained on that data can express. Expert demonstrations, whose narrow action distributions let next states be predicted without attending to the action, set a low ceiling that no amount of model capacity alone can raise. Raising it requires collecting different data. We introduce Action Influence Maximization (AIM), an algorithm for actively collecting training data for improving dataset action influence. We compare the performance and data-efficiency of AIM against exploration heuristics and intrinsic-motivation methods on simulated navigation and manipulation tasks. In pixel-based manipulation, AIM is the only method that consistently improves controllability: 500 actively collected episodes added to 3k demonstrations match a world model trained on 300k expert episodes on end-effector controllability, an $85\times$ reduction in data.