Driving · 2026-10-09
PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving
Jinchang Xu, Hongda Yu, Fengwei Dong, Wenhui Huang, Xi Wei, Yongzhi Liu, Sunan Zhang, Jirao Wang, Chen Lv, Bingbing Li, Guodong Yin, Weichao Zhuang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
PlanWAM is a planning-shaped world action model for end-to-end autonomous driving. It learns a compact recency-aware history representation, then uses a training-only privileged posterior (ground-truth future frames) to shape a future latent via trajectory generation/selection objectives. A prior branch distilled from the posterior predicts this planning-shaped future latent from history at inference, providing foresight for trajectory planning.
Why it matters
The authors argue that future representation learning for driving should be shaped by planning objectives, not generic prediction targets. PlanWAM explicitly trains a future latent that is directly optimized for how it helps trajectory generation and selection, and then distills that privileged target into a deployable history-only prior for open-loop and zero-shot closed-loop driving.
Method
- Temporal Register Pyramid compresses multi-frame, multi-view history in a recency-aware way into a compact history representation for future reasoning and planning.
- Training-only privileged posterior uses ground-truth future scene registers and planning losses (trajectory generation/selection) to produce a planning-shaped future latent; concatenates it with history for planning context.
- Hindsight-to-Foresight Distillation trains a history-only prior to predict the planning-shaped future latent, which is used by the trajectory planner and scorer at inference.
Abstract (from arXiv)
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.
Related papers
- V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving
- MORE: Learning Motion-Oriented Registers for Driving World Models and Planning
- World4Scorer: Outcome-Grounded World Modeling for Autonomous Driving
- OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
- Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.11382
Explicit source versions (1)
- v1 · 2026-10-08
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World models in end-to-end autonomous driving predict future scene evolution to provide foresight for trajectory planning. Existing methods mainly study how to predict the future and how to use it, but less often ask which future representation is actually most useful for planning. To this end, we propose PlanWAM, a Planning-Shaped World Action Model. The key idea is to let the planning task shape the future-state representation, so that it retains the information most useful for planning. A latent world model then predicts this planning-shaped future latent representation from historical observations and uses it for planning, enabling foresighted planning. Specifically, we first use a Temporal Register Pyramid to compress multi-frame historical information in a recency-aware manner, learning a compact history representation oriented toward future reasoning and planning. We then introduce a privileged future posterior branch that observes ground-truth future frames, and shape its future latent representation with trajectory-planning objectives to obtain a planning-shaped future latent representation. Hindsight-to-Foresight Distillation trains a prior branch that depends only on history to predict this future latent representation. The predicted future latent representation serves as planning context and guides trajectory generation and selection. PlanWAM achieves 93.8 PDMS / 90.9 EPDMS on NAVSIM-v1/v2 navtest and reaches 38.7 HD-Score on closed-loop HUGSIM in a zero-shot setting, demonstrating leading planning performance across both open-loop and closed-loop evaluations. Extensive experiments further demonstrate that planning-shaped future representations provide an effective and deployable form of foresight for world-action models.