Driving · 2026-10-20
Train An Imaginative Agent
Jiaxin Ge, Junyi Zhang, Tianning Chai, Hanjun Yoo, Chenyang Wang, Xiuyu Li, Weijia Shi, XuDong Wang, Sewon Min, Luke Zettlemoyer, Trevor Darrell, Haiwen Feng
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Agima (Agentic Imagination) trains an agent to imagine before acting. A single omni model proposes actions and generates imagined images of their outcomes, then reasons over imagined futures to plan. Supervised fine-tuning unifies policy and world modeling; agentic RL trains adaptive planning and branching. The authors report generalization across Maze2D, Sokoban, Pacman, and PushT, including Sokoban 2-box to 4-box transfer under sparse feedback.
Why it matters
The paper targets failures from irreversibility and sparse feedback, motivating planning that anticipates consequences before execution. Agima is designed to unify policy, world modeling, and reward modeling in one omni model, and to support multiple planning traces (linear rollout, counterfactual branching, forward thinking) using predicted observations. The authors’ results highlight both multi-step open-loop planning and improved decision-making via reasoning over alternative imagined futures.
Method
- Train a single omni model (BAGEL backbone) with supervised fine-tuning on interleaved text/image imagination traces so it can propose actions and predict imagined visual outcomes, then reason over those imagined futures for planning.
- Use agentic reinforcement learning with GRPO: generate full planning traces without real environment observations, execute only the final selected actions, and optimize binary task-success rewards to teach adaptive use of imagination.
- Evaluate planning with imagined observations at different feedback intervals K (including K=∞ open-loop) and test counterfactual branching and forward thinking on Sokoban and Pacman reasoning tasks.
Limitation
The real execution can disagree with the model’s outcome assessment; the paper shows examples where the model describes a chosen plan as successful but the executed plan fails.
Abstract (from arXiv)
A seemingly reasonable push in Sokoban can make the puzzle unsolvable. To avoid such mistakes, an agent has to anticipate the consequences of its actions before execution, especially when feedback from the real environment is sparse or actions are irreversible. We introduce Agentic Imagination (Agima), a training framework that trains an agent to imagine before acting, i.e. to be imaginative. Agima trains a single omni model to propose actions, generate images of their outcomes, and then reason over these imagined futures to decide what to execute in the real environment. Rather than committing to a single plan, the agent can simulate a sequence of actions, explore alternatives, and compare possible futures. We first use supervised fine-tuning (SFT) to unify policy and world modeling within a single omni model. This enables the agent to predict future outcomes and perform basic forms of imagination, such as forward thinking and counterfactual branching. We then use agentic reinforcement learning (RL) to teach the agent to reason with its imagined futures and use them effectively for planning. Across four environments spanning Maze2D, Sokoban, Pacman, and PushT, Agima generalizes to harder settings unseen during training. Trained with SFT alone on two-box Sokoban, Agima solves 40% of four-box puzzles while seeing the real board only once every five moves, compared with an action-only planner that only solves 2%. After RL, new behaviors emerge in how Agima uses its imagination: it learns a new unified format that was unseen during the SFT stage, learns to conduct forward thinking only when necessary, and branches more frequently at critical decision points. An agent that can picture where its actions lead can learn when and where to look ahead, that is, to plan. Project page: https://agima.net/.
Related papers
- Latent Imagination for Parameterized Action Spaces
- Second Thoughts: Exploring and Planning through Comparative Imagination in World Models
- Dream to Control: Learning Behaviors by Latent Imagination
- Deploying Physically and Epistemically Compliant Agents by Distilling Belief-Space Task and Motion Planners
- The Planning Limits of Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
Train An Imaginative Agent
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.26845
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
A seemingly reasonable push in Sokoban can make the puzzle unsolvable. To avoid such mistakes, an agent has to anticipate the consequences of its actions before execution, especially when feedback from the real environment is sparse or actions are irreversible. We introduce Agentic Imagination (Agima), a training framework that trains an agent to imagine before acting, i.e. to be imaginative. Agima trains a single omni model to propose actions, generate images of their outcomes, and then reason over these imagined futures to decide what to execute in the real environment. Rather than committing to a single plan, the agent can simulate a sequence of actions, explore alternatives, and compare possible futures. We first use supervised fine-tuning (SFT) to unify policy and world modeling within a single omni model. This enables the agent to predict future outcomes and perform basic forms of imagination, such as forward thinking and counterfactual branching. We then use agentic reinforcement learning (RL) to teach the agent to reason with its imagined futures and use them effectively for planning. Across four environments spanning Maze2D, Sokoban, Pacman, and PushT, Agima generalizes to harder settings unseen during training. Trained with SFT alone on two-box Sokoban, Agima solves 40% of four-box puzzles while seeing the real board only once every five moves, compared with an action-only planner that only solves 2%. After RL, new behaviors emerge in how Agima uses its imagination: it learns a new unified format that was unseen during the SFT stage, learns to conduct forward thinking only when necessary, and branches more frequently at critical decision points. An agent that can picture where its actions lead can learn when and where to look ahead, that is, to plan. Project page: https://agima.net/.
