Latent Dynamics · 2026-10-21
Knowing When to Stop Thinking: A Dissection of Expected Free Energy as a Test-Time Compute Controller for Latent World-Model Planning
Tong Zhang, Zhiyuan Shi, Yun Peng, Tao Xie
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors study when expected-free-energy (EFE) style signals can control test-time compute for latent world-model planning. They evaluate candidate scoring, stopping, and online predictor updates separately. Cost-stall stopping reduces refinement on tasks where success plateaus with depth, but can fail when success peaks and further optimization exploits model errors. Online updates recover part of losses from missing training coverage.
Why it matters
Choosing how long to plan affects both performance and compute. The paper dissects whether EFE-like disagreement and cost-progress signals reliably indicate when extra refinement helps or harms, and whether online model updates fix coverage holes. The results highlight failure modes where internal cost keeps improving after external success peaks.
Method
- Use a latent world model (LeWM) with CEM sampling; score candidates using a goal-distance term plus an uncertainty proxy from stochastic rollout disagreement (sign-controlled by β).
- Implement a stopping controller that halts when elite cost improvement stalls, using an EMA of mean elite cost with minimum depth and patience windows; compare with alternative cost statistics.
- Test online adaptation by removing training data from a required region in TwoRoom and updating the latent predictor online on collected transitions; include matched controls and sham/shuffled-target updates.
Limitation
The paper notes that a cost-internal stopping rule can fail: it may continue refining after external success peaks because predicted cost keeps falling, and stopping signals based only on internal cost do not guarantee optimal depth without a link between internal progress and external success.
Abstract (from arXiv)
A planner using a learned world model can improve its predicted outcome while choosing a worse action. We study when expected free energy (EFE) can help decide how much computation to spend on a plan. An EFE-inspired objective combines goal distance with disagreement among stochastic predictions. We test candidate scoring, stopping, and online predictor updates separately. Stopping when elite cost stalls reduces refinement on tasks whose success plateaus with depth, with mean success close to fixed-depth references. At the larger tested weights, rewarding disagreement lowers success with a frozen model. Penalizing it helps at some task and weight settings but harms others. When training data omit a region needed to reach the goal, online updates recover part of the resulting loss. The tested cost-stall rule can fail when further optimization exploits model errors, because predicted cost keeps falling after task success peaks. Cost progress can guide computation on a plateau, but protecting performance after a peak requires information about the consequences of the chosen actions.
Related papers
- When Low Prediction Error Misleads Planning: Diagnosing Representation, Dynamics, and Decision Failures in Latent World Models
- VPFlow: Value-Predictive Planning with Imperfect World Models for Offline Goal Reaching
- Supervise What Decides Success: Criterion-Aligned Auxiliary Losses for Latent World-Model Planning
- Action-Consequence Alignment for Reliable Planning and Self-Improving in Latent World Models
- One-Step Prediction Fidelity Is a Poor Proxy for Planning Performance in JEPA World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-20 · Indexed by Pub-AI: 2026-10-21 UTC
Knowing When to Stop Thinking: A Dissection of Expected Free Energy as a Test-Time Compute Controller for Latent World-Model Planning
Preprint posted: 2026-10-20 · Original record ↗
- arxiv
- 2610.28011
Explicit source versions (1)
- v1 · 2026-10-20
- Preprint available
Source date: 2026-10-20 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · OAI arXiv created; not header datestamp (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
A planner using a learned world model can improve its predicted outcome while choosing a worse action. We study when expected free energy (EFE) can help decide how much computation to spend on a plan. An EFE-inspired objective combines goal distance with disagreement among stochastic predictions. We test candidate scoring, stopping, and online predictor updates separately. Stopping when elite cost stalls reduces refinement on tasks whose success plateaus with depth, with mean success close to fixed-depth references. At the larger tested weights, rewarding disagreement lowers success with a frozen model. Penalizing it helps at some task and weight settings but harms others. When training data omit a region needed to reach the goal, online updates recover part of the resulting loss. The tested cost-stall rule can fail when further optimization exploits model errors, because predicted cost keeps falling after task success peaks. Cost progress can guide computation on a plateau, but protecting performance after a peak requires information about the consequences of the chosen actions.