Driving · 2026-11-10

Latent-WAM++: Shaping World and Action Representations for End-to-End Autonomous Driving

Yichen Zhang, Yupeng Zheng, Linbo Wang, Pengxuan Yang, Zebin Xing, Junli Wang, Shiwei Li, Kangwei Liu, Fan Ren, Qichao Zhang, Yifeng Pan, Ji Tao, Dongbin Zhao

arXiv:2611.10632PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

Latent-WAM++ is a representation-centered World Action Model for end-to-end autonomous driving. It jointly shapes world and action representations by (1) constructing a temporally predictive, non-degenerate, geometry-aware world representation using Scene-Ego latent prediction + SIGReg + frozen geometric priors (without an EMA target encoder), (2) a one-step multimodal Trajectory MeanFlow planner, and (3) Visual-Action Representation Alignment with a stop-gradient visual target. The authors report SOTA on NAVSIM v2 and zero-shot on HUGSIM.

Why it matters

The paper argues that in prior WAMs, intermediate world and action representations are only indirectly constrained by output-oriented objectives, which can lead to feature collapse and misalignment between planning-relevant representations. By matching supervision to each intermediate representation’s planning role—stabilizing world latents with SIGReg, directly parameterizing trajectory action features with one-step MeanFlow, and aligning action features to visual semantics—the authors aim to improve both prediction quality and planning outcomes for autonomous driving.

Method

  • World Representation Construction (WoRC): scene-ego future latent prediction with SIGReg-based distribution stabilization (target-free, no EMA target encoder) plus frozen geometric priors.
  • Trajectory MeanFlow Planner (TrajMF): conditional flow-matching that predicts and aggregates multimodal trajectory proposals in one step, optimized with an Improved MeanFlow velocity-space objective.
  • Visual-Action Representation Alignment (VARA): aligns pre-output action features with a stop-gradient target derived from current visual semantics via an asymmetric cosine objective.
Abstract (from arXiv)

We introduce Latent-WAM++, a representation-centered World Action Model (WAM) for end-to-end autonomous driving. Existing WAMs are primarily trained to predict future states or final actions, leaving the intermediate world and action representations that connect prediction to planning only indirectly constrained. Latent-WAM++ matches the supervision of each representation to its role in planning. World Representation Construction combines scene-ego future prediction, SIGReg-based distribution stabilization, and frozen geometric priors to learn a temporally predictive, non-degenerate, and geometry-aware world representation without an EMA target encoder. Trajectory MeanFlow Planner directly predicts multimodal trajectory proposals while being optimized with an Improved MeanFlow velocity-space objective, enabling one-step sampling. Visual-Action Representation Alignment further aligns the resulting pre-output action features with a stop-gradient target derived from current visual semantics. Latent-WAM++ achieves state-of-the-art performance on NAVSIM v2 and under zero-shot evaluation on HUGSIM, reaching 90.1 EPDMS on NAVSIM v2 and 47.0 average RC and 30.1 average HD-Score on HUGSIM. Training the planner end to end costs 190 GPU-hours for Latent-WAM++ versus 1,600 for Latent-WAM, a reduction of 88.1%.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

Latent-WAM++: Shaping World and Action Representations for End-to-End Autonomous Driving

Yichen Zhang, Yupeng Zheng, Linbo Wang, Pengxuan Yang, Zebin Xing, Junli Wang, Shiwei Li, Kangwei Liu, Fan Ren, Qichao Zhang, Yifeng Pan, Ji Tao, Dongbin Zhao

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.10632
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.10632v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

We introduce Latent-WAM++, a representation-centered World Action Model (WAM) for end-to-end autonomous driving. Existing WAMs are primarily trained to predict future states or final actions, leaving the intermediate world and action representations that connect prediction to planning only indirectly constrained. Latent-WAM++ matches the supervision of each representation to its role in planning. World Representation Construction combines scene-ego future prediction, SIGReg-based distribution stabilization, and frozen geometric priors to learn a temporally predictive, non-degenerate, and geometry-aware world representation without an EMA target encoder. Trajectory MeanFlow Planner directly predicts multimodal trajectory proposals while being optimized with an Improved MeanFlow velocity-space objective, enabling one-step sampling. Visual-Action Representation Alignment further aligns the resulting pre-output action features with a stop-gradient target derived from current visual semantics. Latent-WAM++ achieves state-of-the-art performance on NAVSIM v2 and under zero-shot evaluation on HUGSIM, reaching 90.1 EPDMS on NAVSIM v2 and 47.0 average RC and 30.1 average HD-Score on HUGSIM. Training the planner end to end costs 190 GPU-hours for Latent-WAM++ versus 1,600 for Latent-WAM, a reduction of 88.1%.