Driving · 2026-10-07
RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation
Jing Xie, Shouwei Ruan, Yubin Wang, Yuxiang Zhang, Haitao Yang, Songchang Jin, Dianxi Shi
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
RIWANav is a post-training framework that recursively improves a navigation policy and a visual action-conditioned world model for long-horizon urban navigation. Each recursion alternates (1) world-model-based imagined outcome comparison for GRPO policy optimization and (2) policy-guided construction of a grounded self-curriculum of expert-consistent action–video pairs (by behavioral novelty and prediction error) to refine the world model. Experiments on UrbanNav show consistent gains over baselines; real-world trials demonstrate feasibility.
Why it matters
The authors address a bottleneck in model-based post-training: a frozen world model can become misaligned as the policy evolves, distorting subsequent policy updates. RIWANav closes this loop by coupling world-model adaptation to policy-induced data selection, using imagined outcomes for policy improvement and a grounded self-curriculum for world-model refinement. This targets compounding errors in long-horizon urban navigation while reducing costly physical trial-and-error.
Method
- Recursive self-improvement loop: each cycle alternates world-model evaluation of candidate actions via imagined visual outcomes for GRPO, then world-model adaptation using policy-induced action–video pairs.
- Grounded self-curriculum: admit only expert-consistent candidates paired with recorded future frames; prioritize accepted samples using behavioral novelty relative to the initial policy and current prediction error.
- World-simulator policy optimization: sample multiple candidate actions from the policy, score them by imagined consequence quality (group-relative advantage), and use expert anchor losses and KL constraints to stabilize updates.
Limitation
The authors state the current implementation remains task-specific and relies on a fixed offline dataset; recorded futures only supervise policy actions close to expert motion, limiting counterfactual exploration without new interaction data. They also note that adapting a large video world model increases memory and computation.
Abstract (from arXiv)
Long-horizon urban navigation requires sequential local decisions whose errors can compound over time. Imitation learning (IL) rarely learns from failures, while physical trial-and-error reinforcement learning (RL) is costly. Action-conditioned world models can provide imagined feedback by predicting visual consequences for candidate actions. However, a frozen world model may become less reliable as the policy evolves. In this paper, we introduce RIWANAV, a post-training framework that casts the coupled adaptation of a world model and an action model (policy) as task-specific recursive self-improvement (RSI). Each cycle alternates two updates. The world model evaluates policy actions through imagined outcomes, providing comparative feedback for group-relative policy optimization (GRPO). The improved policy then constructs a grounded self-curriculum, selecting expert-consistent action-video pairs by behavioral novelty and prediction error. The refined world model supplies feedback for the next policy update, closing the recursive self-improvement loop. Experiments show that RIWANAV outperforms training baselines and prior methods, validating the proposed recursive self-improvement loop between the policy and world model. Real-world trials further demonstrate its practical applicability.
Related papers
- PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
- DRILL: Training World Models to Improve Policies, Not Predict Pixels
- Training Generalist Navigation Policies via Action Self-Distillation
- World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning
- Memento 3: Model-Based Recursive Self-Improvement through Reflective Rulebooks
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-06 · Indexed by Pub-AI: 2026-10-07 UTC
RIWANav: Recursive World-Action Models with Self-Improvement for Urban Navigation
Preprint posted: 2026-10-06 · Original record ↗
- arxiv
- 2610.08640
Explicit source versions (1)
- v1 · 2026-10-06
- Preprint available
Source date: 2026-10-06 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Long-horizon urban navigation requires sequential local decisions whose errors can compound over time. Imitation learning (IL) rarely learns from failures, while physical trial-and-error reinforcement learning (RL) is costly. Action-conditioned world models can provide imagined feedback by predicting visual consequences for candidate actions. However, a frozen world model may become less reliable as the policy evolves. In this paper, we introduce RIWANAV, a post-training framework that casts the coupled adaptation of a world model and an action model (policy) as task-specific recursive self-improvement (RSI). Each cycle alternates two updates. The world model evaluates policy actions through imagined outcomes, providing comparative feedback for group-relative policy optimization (GRPO). The improved policy then constructs a grounded self-curriculum, selecting expert-consistent action-video pairs by behavioral novelty and prediction error. The refined world model supplies feedback for the next policy update, closing the recursive self-improvement loop. Experiments show that RIWANAV outperforms training baselines and prior methods, validating the proposed recursive self-improvement loop between the policy and world model. Real-world trials further demonstrate its practical applicability.