Driving · 2026-11-06

GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving

Xiyan Su, Finn Rasmus Sch\"afer, Hojun Lim, Frank Diermeyer, Johannes Betz, Markus Lienkamp

arXiv:2611.07912PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

GuidedWAM is a model-agnostic, training-free energy guidance method that injects new driving objectives into frozen world action models at sampling time. It constructs objectives as differentiable energies evaluated on each sampler step’s one-step clean-action estimate and applies gradients to the sampler’s velocity update. On NAVSIM v2 hard-split, guidance improves three frozen, publicly released WAMs by 1.8–3.6 EPDMS with <10% sampling-time overhead per step.

Why it matters

WAMs typically freeze their driving objectives after training and require retraining for new objectives. The authors propose a post-training control channel that uses only forward access to a frozen model, enabling objective changes at deployment time without retraining. They show measurable improvements on NAVSIM v2 hard-split across three public WAMs, with guidance stabilization techniques and analysis of computation overhead.

Method

  • Construct each driving objective as a differentiable energy on the sampler’s one-step clean estimate of the action sequence (decoded to ego trajectory).
  • Apply energy guidance by adding an analytic gradient of the energy (evaluated on a detached clean estimate) to the frozen flow-matching sampler’s velocity field, without backpropagating through the frozen denoiser.
  • Stabilize sampling-time guidance with scheduling (trapezoidal noise-level window), per-energy gradient normalization, and velocity-bounded correction.

Limitation

Our evaluation is open-loop pseudo-simulation on one benchmark, with no closed-loop test. Since the guidance is soft, the energy terms can only reduce violations but not guarantee that a driving objective is enforced.

Abstract (from arXiv)

World action models (WAMs) for autonomous driving couple the driving policy and the visual world model in a single generative network whose objectives are fixed post-training. Conditioning on a new driving objective requires retraining. We study energy guidance as a general mechanism and present GuidedWAM, a model-agnostic framework for injecting driving objectives into frozen WAMs at sampling time. Our framework constructs each objective as differentiable energies on the sampler's intermediate clean-action estimate, which needs only forward access to the frozen model, and is stabilized by sampling-time guidance scheduling, per-energy normalization, and velocity-bounded correction. On the NAVSIM v2 hard-split, guidance improves all three frozen, publicly released WAMs by 1.8 to 3.6 EPDMS, averaged over three sampler seeds, and adds less than 10% to the sampling time per step. These results position energy guidance as a practical post-training control channel for WAMs.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-06 · Indexed by Pub-AI: 2026-11-06 UTC

preprint

GuidedWAM: Training-Free Energy Guidance for World Action Models in Autonomous Driving

Xiyan Su, Finn Rasmus Schäfer, Hojun Lim, Frank Diermeyer, Johannes Betz, Markus Lienkamp

First announced by arXiv: 2026-11-06 · Original record ↗

arxiv
2611.07912
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-06 · First detected: 2026-11-06 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.07912v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

World action models (WAMs) for autonomous driving couple the driving policy and the visual world model in a single generative network whose objectives are fixed post-training. Conditioning on a new driving objective requires retraining. We study energy guidance as a general mechanism and present GuidedWAM, a model-agnostic framework for injecting driving objectives into frozen WAMs at sampling time. Our framework constructs each objective as differentiable energies on the sampler's intermediate clean-action estimate, which needs only forward access to the frozen model, and is stabilized by sampling-time guidance scheduling, per-energy normalization, and velocity-bounded correction. On the NAVSIM v2 hard-split, guidance improves all three frozen, publicly released WAMs by 1.8 to 3.6 EPDMS, averaged over three sampler seeds, and adds less than 10% to the sampling time per step. These results position energy guidance as a practical post-training control channel for WAMs.