Driving · 2026-10-14
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
Yi Ge, Kangrui Ruan, Jifan Chen, Manish Prajapat, Christian Bock, Xiaoyang Wang, Rein Houthooft, Aritra Sengupta, Varun Kumar
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ANTE is an agentic RL framework for coding agents that explicitly writes a belief state before each action, including an outcome forecast and a trust/abstain decision. During RL, Belief-Anchored Policy Optimization (BAPO) verifies forecasts by real execution and turns them into per-step credit. At inference, ANTE-TTO samples and ranks candidate actions using the learned forecasts.
Why it matters
The authors argue most coding agents do not anticipate action consequences and receive only outcome-level terminal rewards, which can allow repository-corrupting edits before any signal arrives. ANTE aims to combine explicit, execution-verified forecasts with per-step credit assignment and test-time action selection, to improve both prediction accuracy and task success.
Method
- Before each tool call, the policy writes a belief state containing a one-sentence summary, an outcome forecast read from execution, and a trust/abstain decision.
- During RL, ANTE verifies each forecast by real execution and uses BAPO to group steps with the same predicted outcome and assign per-step credit beyond final task reward.
- At inference, ANTE-TTO samples alternative actions when the policy abstains or predicts failure, ranks candidates by its own forecasts, and executes the highest-ranked action without an external scorer.
Abstract (from arXiv)
Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.
Related papers
- Arctic-SWE: Revisiting the Long Horizon Agent Training by Modeling-System Co-Design
- EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
- ObsSpec: Accelerating Agentic Reinforcement Learning with Observation Speculation
- Failure Recovery via Belief Updates and World Modeling for Long-Horizon Agents
- Self-Improving Agents with World Model Predictive Paths
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-11 · Indexed by Pub-AI: 2026-10-14 UTC
ANTE: Anticipate, Then Execute -- Learning Internal World Models for Coding Agents from Verified Beliefs
Preprint posted: 2026-10-11 · Original record ↗
- arxiv
- 2610.15368
Explicit source versions (1)
- v1 · 2026-10-11
- Preprint available
Source date: 2026-10-11 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Rapid advances in large language models have made software engineering agents capable of tackling complex coding tasks. Within a containerized environment, a coding agent receives tool responses or execution results, and determines its later actions by those earlier results. However, most existing coding agents never look ahead: their policies do not anticipate the consequences of actions, and the terminal reward barely attributes a failure to certain earlier steps, so an ill-judged edit can corrupt the repository state irreversibly before any signal arrives. Recent code world models aim to address these challenges, but they are either trained and then frozen before reinforcement learning (RL), or reduced to an auxiliary next-token loss on the environment's outputs. We present ANTE (Anticipate, Then Execute), an agentic RL framework that verifies the policy's pre-action predictions by real execution and turns them into per-step credit for its actions and trust-or-abstain decisions. With an internal belief stated before each action, Belief-Anchored Policy Optimization (BAPO) turns the settled predictions into per-step advantages by anchoring credit on the policy's own forecasts. To turn the trained forecasts into better actions at inference, thereby facilitating effective test-time scaling, ANTE-TTO (test-time optimization) selects among candidate actions without an external scorer. ANTE raises Qwen3-32B from 25.0% to 57.2% pass@1 on SWE-bench Verified, 6.2% above a matched GRPO baseline trained without the belief state; its world model's accuracy climbs from 44% to 69% during RL, and the learned foresight capability transfers across harnesses and out-of-distribution domains. In summary, we aim to build a code world model that knows what has happened, what holds now, and what the next action will achieve.