Latent Dynamics · 2026-10-06
Considering Context: When World Models Need Context Encoders
Oleg Smirnov, Sofiane Ennadir, John Pertoft, Bjartur Hjaltason, Sara Karimi
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study when latent-context world models need a context encoder. They formalize “predictive sufficiency” as the Bayes one-step log-loss advantage of conditioning on context under the visitation distribution induced by an agent. They decompose this advantage into headroom recoverable from trajectory history and a residual requiring the true context, and show that benefits need not match MDP classes; implicit identification can make predictive headroom vanish for some priors and persist (partially) for others.
Why it matters
Context-aware model-based RL often assumes the agent cannot infer latent dynamics context from its own experience and therefore supplies context externally or via learned encoders. The paper provides a criterion—predictive sufficiency and its decomposition—that diagnoses whether a given conditioning mechanism has predictive information to exploit under the agent’s induced visitation distribution, separating information availability from finite-model error.
Method
- Define predictive sufficiency as preservation of trajectory transition information about the latent context without requiring recovery of the context itself; relate it to Bayes-optimal one-step log-loss reduction under the agent’s visitation distribution.
- Decompose predictive headroom into history-recoverable information (ℐ_hist) and residual oracle headroom requiring the true context, and account for finite-model estimation/approximation/optimization gaps.
- Run controlled diagnostics in MiniGrid variants and Point-Mass-Attractor to test how visitation-mediated implicit identification changes the estimated predictive headroom, and validate across DreamerV3-based method classes.
Abstract (from arXiv)
Methods for generalization in model-based reinforcement learning typically assume that an agent cannot recover the latent context governing the environment dynamics from its own experience, and therefore supplies it externally. We formalize and test this assumption with \emph{predictive sufficiency}, which quantifies what access to the context adds to next-step prediction under the visitation distribution an agent induces, and separates that quantity into a history-recoverable part, a residual requiring the true context, and the deficit added by a finite model. We classify context-aware algorithms by the predictive risk their conditioning set can target and demonstrate across environments of increasing identification difficulty that the headroom does not follow the MDP class. The same task under different priors leaves predictive headroom in one setting and nothing distinguishable from zero in another, where the agent's behavior implicitly identifies the context and any benefit of such a mechanism cannot be attributed to missing information. Where headroom persists, the learned state exposes it only partially, and adding the true context still lowers the risk. Our contribution is a practical criterion for matching contextual mechanisms to the information available to them, estimated from the ordinary trained agent without a reference policy.
Related papers
- Shaping Persistent Representations from Independent Interactions
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling
- CausalDreamer: Learning Predictive World Models with Latent Disentanglement
- MemoWM: How World Models Change What Agents Need to Remember
- An Introduction to World Models for Generalist Agents
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-05 · Indexed by Pub-AI: 2026-10-06 UTC
Considering Context: When World Models Need Context Encoders
Preprint posted: 2026-10-05 · Original record ↗
- arxiv
- 2610.06651
Explicit source versions (1)
- v1 · 2026-10-05
- Preprint available
Source date: 2026-10-05 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Methods for generalization in model-based reinforcement learning typically assume that an agent cannot recover the latent context governing the environment dynamics from its own experience, and therefore supplies it externally. We formalize and test this assumption with \emph{predictive sufficiency}, which quantifies what access to the context adds to next-step prediction under the visitation distribution an agent induces, and separates that quantity into a history-recoverable part, a residual requiring the true context, and the deficit added by a finite model. We classify context-aware algorithms by the predictive risk their conditioning set can target and demonstrate across environments of increasing identification difficulty that the headroom does not follow the MDP class. The same task under different priors leaves predictive headroom in one setting and nothing distinguishable from zero in another, where the agent's behavior implicitly identifies the context and any benefit of such a mechanism cannot be attributed to missing information. Where headroom persists, the learned state exposes it only partially, and adding the true context still lowers the risk. Our contribution is a practical criterion for matching contextual mechanisms to the information available to them, estimated from the ordinary trained agent without a reference policy.