Driving · 2026-10-15
SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving
Shaoyuan Xie, Zihan Zhang, Jingxuan Wang, Jiashu Qu, Xiaoqing Liang, Lingdong Kong, Junchi Lu, Henrik I. Christensen, Qi Alfred Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SafeDriveVLA is a driving Vision-Language-Action model for conflict-aware navigation following. The authors propose (1) predicting a three-level navigation-compliance mode (strict/cautious/fallback) to decouple conflict reasoning from action generation, and (2) navigation-conditioned world model dreaming to inform this mode prediction. They evaluate on Bench2Drive and their new benchmark suite and report improved driving performance and fewer collisions under unsafe instructions.
Why it matters
The authors target a vulnerability of end-to-end driving VLAs: open-ended navigation instructions can be unsafe given the current scene. SafeDriveVLA adds explicit navigation-compliance mode prediction and uses navigation-conditioned dreaming to indicate when following is safe, evaluated with benchmarks spanning navigation following, navigation-scene conflict awareness, and adversarial instructions.
Method
- Relabel expert trajectories with a discrete driving mode (strict/cautious/fallback) indicating whether the trajectory executes, cautiously follows, or overrides the navigation signal; predict a mode token before action tokens.
- Train an action-conditioned latent world model (from V-JEPA 2 AC encoder/predictor) and freeze it so that at inference a navigation-anchored rollout produces dreamed latents of strict execution.
- Autoregressively generate mode token and action tokens conditioned on the camera frame, navigation signal, and dreamed latents (plus an auxiliary path head for lateral control).
Abstract (from arXiv)
Vision-Language-Action (VLA) models are gaining attention in autonomous driving, inheriting commonsense priors from large language models (LLMs) and widening the navigation interface from a fixed command set to free-form natural-language instructions. However, this open-ended interface exposes a new vulnerability surface: an inattentive driver may issue instructions that are unsafe given the current scene, and a malicious user may issue dangerous ones on purpose. We present the first benchmark suite that evaluates navigation following and conflict awareness together, along with adversarial robustness. Existing VLAs fall short on both: SimLingo completes no language-instructed lane change in closed loop, and all models degrade sharply under conflicting signals. We propose SafeDriveVLA, the first driving VLA that (1) explicitly predicts a three-level navigation-compliance mode to decouple conflict reasoning from action generation, and (2) uses navigation-conditioned world model dreaming to inform this prediction. Unlike prior world models that roll out expert or planned actions, ours dreams of strictly following the navigation signal, showing when following is safe. SafeDriveVLA reaches 88.16 Driving Score (DS) and 68.64% Success Rate (SR) on Bench2Drive, lifts closed-loop language following by 49.3% and reduces collisions under unsafe instructions from 60 to 38 compared to the best SimLingo variant. Video demonstrations are available on our project page.
Related papers
- Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?
- AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding
- doPlan: A Variable-Horizon Dataset for Multi-Stage Language-Conditioned Planning in Autonomous Driving
- Scene-Adaptive VLA: Efficient Autonomous Driving via Dynamic Layer Routing
- VGGTWorld-VLA: Intent-Conditioned 3D World Evolution for Autonomous Driving
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-15 UTC
SafeDriveVLA: Navigation-Conditioned World Model Dreaming for Conflict-Aware End-to-End Autonomous Driving
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20609
Explicit source versions (1)
- v1 · 2026-10-14
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) models are gaining attention in autonomous driving, inheriting commonsense priors from large language models (LLMs) and widening the navigation interface from a fixed command set to free-form natural-language instructions. However, this open-ended interface exposes a new vulnerability surface: an inattentive driver may issue instructions that are unsafe given the current scene, and a malicious user may issue dangerous ones on purpose. We present the first benchmark suite that evaluates navigation following and conflict awareness together, along with adversarial robustness. Existing VLAs fall short on both: SimLingo completes no language-instructed lane change in closed loop, and all models degrade sharply under conflicting signals. We propose SafeDriveVLA, the first driving VLA that (1) explicitly predicts a three-level navigation-compliance mode to decouple conflict reasoning from action generation, and (2) uses navigation-conditioned world model dreaming to inform this prediction. Unlike prior world models that roll out expert or planned actions, ours dreams of strictly following the navigation signal, showing when following is safe. SafeDriveVLA reaches 88.16 Driving Score (DS) and 68.64% Success Rate (SR) on Bench2Drive, lifts closed-loop language following by 49.3% and reduces collisions under unsafe instructions from 60 to 38 compared to the best SimLingo variant. Video demonstrations are available on our project page.