Video World Models · 2026-10-09
WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
Ankan Deria, Komal Kumar, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
WorldGuide formulates procedural video generation as closed-loop task execution in visual world space. Given an initial image and task goal, a ContextPlanner predicts the next atomic action or a DONE token from the generated visual state and recent clip-action history; a diffusion-transformer Executor renders the corresponding next video clip using hierarchical visual memory. The model is trained on step-level procedural demonstrations from WorldGuide Bench.
Why it matters
The authors target a gap between selecting actions and reliably realizing them in long-horizon procedural tasks. They argue open-loop generation cannot adapt to execution outcomes, while many closed-loop systems rely on frozen or indirectly verified executors. By coupling step-level planning with directly trained action-conditioned execution and learned termination, WorldGuide improves task completion when only the initial image and goal are provided.
Method
- Goal-directed closed-loop rollout: ContextPlanner predicts next atomic action or DONE from goal, current visual state, and last K clip-action pairs; Executor renders the next clip conditioned on the predicted action and hierarchical visual memory.
- Planner–Executor training on the same step-level procedural demonstrations: Planner learns next-action/completion prediction; Executor learns to realize the predicted atomic actions under direct supervision.
- WorldGuide Bench provides paired ordered atomic-action clips, task goals, and explicit completion supervision: 58,679 videos across 245 tasks and 27 categories.
Limitation
The ContextPlanner may repeat an action, skip a prerequisite, or stop too early; the Executor may distort geometry or appearance even for a correct instruction. Errors compound across steps and later replanning cannot undo an incorrect transition. The two modules are trained sequentially, without a joint task-completion objective, and bounded memory can discard details needed later.
Abstract (from arXiv)
Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced. A model must determine the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification. This leaves a gap between deciding an action and successfully realizing it. We formulate procedural video generation as \emph{closed-loop task execution in visual world space} and introduce \textbf{WorldGuide}. Given only an initial image and a task goal, WorldGuide predicts an atomic action, generates its corresponding video clip, and uses the generated result to select the next action or terminate. The Planner and Executor are trained on the same step-level procedural demonstrations: the Planner learns to predict the next atomic action or task completion from visual progress, while the Executor is directly trained to realize the predicted actions. Hierarchical visual memory maintains state across long-horizon execution with bounded history token cost. Due to the lack of step-level action-video supervision for joint planner-executor training, we introduce \textbf{WorldGuide Bench}: approximately 59K step-annotated videos across 245 tasks and 27 procedural categories. WorldGuide achieves a 33.33\% Task Success on \textbf{WorldGuide-Bench}, compared with 29.90\% for the strong recent video model MiniMax-H3, even though MiniMax-H3 receives reference action plans, and achieves 47.69\% on \textbf{VideoCraft-Bench} compared with 32.73\% for MiniMax-H3 under goal-only conditioning. These results demonstrate the importance of coupling planning with learned execution for goal-directed procedural video generation.
Related papers
- WorldBridge: Bridging Video and Action for Task Generalization in World Action Models
- WorldLine: Action-Driven Visual Simulation for Robotic Manipulation
- World Action Modeling with Progressive Visual Planning
- CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
- TrajWAM: An Efficient Trajectory-Guided World Action Model for Dynamic Robotic Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
WorldGuide: Goal-Directed Video World Model for Procedural Task Execution
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.12459
Explicit source versions (1)
- v1 · 2026-10-08
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced. A model must determine the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification. This leaves a gap between deciding an action and successfully realizing it. We formulate procedural video generation as \emph{closed-loop task execution in visual world space} and introduce \textbf{WorldGuide}. Given only an initial image and a task goal, WorldGuide predicts an atomic action, generates its corresponding video clip, and uses the generated result to select the next action or terminate. The Planner and Executor are trained on the same step-level procedural demonstrations: the Planner learns to predict the next atomic action or task completion from visual progress, while the Executor is directly trained to realize the predicted actions. Hierarchical visual memory maintains state across long-horizon execution with bounded history token cost. Due to the lack of step-level action-video supervision for joint planner-executor training, we introduce \textbf{WorldGuide Bench}: approximately 59K step-annotated videos across 245 tasks and 27 procedural categories. WorldGuide achieves a 33.33\% Task Success on \textbf{WorldGuide-Bench}, compared with 29.90\% for the strong recent video model MiniMax-H3, even though MiniMax-H3 receives reference action plans, and achieves 47.69\% on \textbf{VideoCraft-Bench} compared with 32.73\% for MiniMax-H3 under goal-only conditioning. These results demonstrate the importance of coupling planning with learned execution for goal-directed procedural video generation.