Driving

World models for autonomous driving. (472 papers)

2026-11-13 · arXiv:2611.14815

BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model

Junyoung Seo, Jaewoo Seo, Seungryong Kim et al.

BirdDreams is an autoregressive driving world model that jointly generates perspective video and a schematic metric bird’s-eye-view (BEV) video. It trains on synchronized RGB–BEV pairs rendered from annotations and uses the predicted BEV as a geometric interface to condition ego-trajectory rollouts without HD map or vehicle-state inputs at inference.

DrivingAI summary
2026-11-13 · arXiv:2611.14955

The Posterior-Prior Epistemic Gap: When World Model Imagination Fails to Carry Task-Relevant Signals

Jaerock Kwon, Elahe Delavari, Donghyun Kim et al.

The authors report a measured posterior-prior epistemic gap in an RSSM world model with active-inference planning for CARLA driving: an obstacle probe scores AUC 0.85 on observation-filtered posterior states, but decays on imagined prior states to AUC 0.71 after 15 imagination steps (57% logit-separation decay). Obstacle avoidance stays at 0% without privileged runtime coordinates and ensemble epistemic disagreement fails to flag the loss.

DrivingAI summary
2026-11-13 · arXiv:2611.15009

From Forecasting to World Modeling: Can Temporal Foundation Models Recover Causal Response?

Juwon Hong, Minha Hwang, Venkatesh Shankar

The authors evaluate released temporal and causal foundation models for retail demand, asking whether good forecasting implies correct causal response to price changes. They report that forecast accuracy does not indicate accuracy of predicted price effects (on simulated data with ground truth and on real sales direction). Extra context improves forecasts but not counterfactual price response under confounding. Instrumental variables help only when supplied in a specific form (residuals).

DrivingAI summary
2026-11-13 · arXiv:2611.15011

Fast Inference-Time Policy Refinement via Action-Affine World Models

Run Wang, Yilin Mo, Liang Wu

The authors propose an action-affine world model (A²WM) for fast inference-time refinement of a frozen imitation policy. Given interaction history and a policy action proposal, A²WM predicts a nominal trajectory plus a linear response to action corrections over the full horizon. This enables quadratic-program (QP) refinement with one model evaluation and one QP solve per step, without updating policy/model weights. Across four simulated manipulation tasks, it reports the highest success rate among evaluated refinement baselines.

DrivingAI summary
2026-11-13 · arXiv:2611.15238

Towards Robust Vision-Language Reasoning: A Multi-Stage Optimization Pipeline with Masked Diffusion Model

Shunming Liu, Pei Huang, Yajing Sun et al.

The authors propose a multi-stage optimization pipeline for discrete diffusion vision-language models in autonomous driving, covering (1) trajectory discretization in supervised fine-tuning to suppress hallucinations, (2) metric-guided diversity filtering during reinforcement learning to improve exploration, and (3) Dynamic Autonomous Driving Decoding (DAD) at inference to adapt decoding order by scene complexity. They develop Masked Diffusion Drive (MD-Drive).

DrivingAI summary
2026-11-13 · arXiv:2611.15260

Driving World Models Move the Camera, Not the Scene

Yuqiao Feng, Xiaoyu Ao

The authors report that plan-conditioned driving world models typically move the camera but barely move the scene: their scene response to speed or lateral plans reaches at most about a third of real-video response (and at most a quarter for a 0.54 m lateral shift). The trajectory ADE score can’t distinguish executing vs ignoring speed commands due to scale alignment. They propose ReTime-Compose, which delivers the plan as geometry without training.

DrivingAI summary
2026-11-13 · arXiv:2611.15296

APG-Planner: Learning Vision-Based High-Speed Trajectory Planning via Analytic Policy Gradient

Jinwoo Lee, Geunsik Bae, Changseung Kim et al.

APG-Planner is a vision-based high-speed trajectory planner trained with analytic policy gradients (APG). The method uses differential flatness to analytically reconstruct vehicle states from planned trajectories, enabling differentiable trajectory-level state transitions without explicitly modeling low-level dynamics. The resulting differentiable episode rollout supports backpropagation through the planning sequence and avoids noisy sample-based policy-gradient estimation. After ~30 minutes of training on a single GPU, it reaches up to 11 m/s in real flight.

DrivingAI summary
2026-11-13 · arXiv:2611.15360

CARE-Drive: Coupled Alternating Refinement for Ego-trajectories

Xiao Ma, Yijun Chen, Zhiquan Hu et al.

CARE-Drive (Coupled Alternating Refinement for Ego-trajectories) refines ego trajectories by separating waypoint residual correction from position-conditioned heading generation. A future-token auxiliary task trains a predictive VLA backbone; then, with the backbone frozen, two specialists are optimized alternately (supervised initialization + reinforcement fine-tuning). On NAVSIM v1, the authors report 88.8 PDMS, +2.4 over the unrefined backbone.

DrivingCode availableAI summary
2026-11-13 · arXiv:2611.15368

Bind-WAM: Binding-Uncertainty-Guided Selective Future Imagination in World Action Models

Chenhao Qiu, Shuyi Chen, Songen Gu et al.

Bind-WAM is a world action model that imagines future only when needed. It uses language-object binding uncertainty—supervised by an offline labeling pipeline that detects reference mismatch, invisibility, and identity drift—to route between direct action prediction and joint future-action denoising. Simulation and real-world experiments show binding uncertainty as an interpretable signal for selective future imagination, with reduced task completion time.

DrivingAI summary
2026-11-13 · arXiv:2611.15689

What World Model Does Active Object Search Need? A Study under Imperfect Detection

Kyeongryeol Go

The authors introduce Bayesian Active Miss-aware Object Search (BAMOS), a belief-based world model for active object search under imperfect detection. It predicts detection likelihood (not full scene reconstruction) using a target belief (location+size), an occupancy belief (for visibility), and fitted detector error rates. Ablations in Isaac Sim and HSSD show target belief and its update mechanisms are critical, with contributions growing as detector errors increase. Code is available.

DrivingAI summary
2026-11-13 · arXiv:2611.16422

BlinkMimic: Training a Humanoid Motion Tracker in Minutes on a Single GPU

Fuyi Chen

BlinkMimic is a single-GPU training recipe for whole-body humanoid motion tracking, building on SuperMimic and InstantTrack. On simulated Unitree G1, the authors report one policy for all 40 LAFAN1 clips reaches a target of mean normalized return ≥0.82 with zero failed clips over three consecutive checkpoints in 3.4 minutes on an RTX 4090. The paper attributes speedups to Muon updates, a progress-triggered step-size schedule, residual actions, deployment-failure probing, and CUDA graph capture of whole iterations.

DrivingCode availableAI summary
2026-11-13 · arXiv:2611.16424

PhysCapBench: A Cross-Domain Benchmark for Physically Grounded Video Captioning

Haofeng Wang, Pengcheng Wen, Ruiqi Li et al.

PhysCapBench is a cross-domain benchmark for physically grounded video captioning across third-person scenes, egocentric manipulation, and handheld-gripper demonstrations. It provides human-verified annotations for 1,500 video sequences and evaluates captions using seven physical dimensions. The authors report that physical captions reduce reconstruction LPIPS by 8.6–9.5% (third-person) and 26.9–30.9% (egocentric) vs metadata captions, and instruction enrichment yields higher PhysCapBench scores and policy success (e.g., +7.5 percentage points for π0.5 on a RoboTwin 2.0 subtask).

DrivingAI summary
2026-11-13 · arXiv:2611.16579

ActWorld-JEPA: Action-Conditioned JEPA World Models Meet Latent Planning for End-to-End Autonomous Driving

Ziying Song, Shengkai Zhang, Caiyan Jia et al.

ActWorld-JEPA is an action-conditioned JEPA world model for candidate-based end-to-end autonomous driving. It predicts candidate-specific latent futures with Plan-Conditioned JEPA (PC-JEPA) and uses Relational Residual Reranking (R3) to refine base trajectory scores via plan–future agreement/discrepancy. The authors report best PDMS on NAVSIM v1 and strong NAVSIM v2 and nuScenes open-loop planning results.

DrivingAI summary
2026-11-13 · arXiv:2611.16767

Evaluating Video LLMs' Understanding of Corner Cases in Autonomous Driving

Gabriel Manalu, Minh Son Hoang, Hyeontaek Hwang et al.

The authors report DashBench, a benchmark for evaluating 25 video LLMs on 1,709 ego-view dashcam clips spanning 88 scenario-level corner-case categories, balanced across collision and non-collision hazards. Using multi-prompt, open-ended QA, the authors find a capability gap: even the top model reaches 46.52% accuracy and models often miss dangerous events and hallucinate nonexistent agents/actions. Code/data are shared at cornercase-dashbench.github.io.

DrivingAI summary
2026-11-11 · arXiv:2611.13030

APS-VLA: Action-Guided and Future-Aware Token Pruning for Efficient Vision-Language-Action Models

Xiaoyang Xing, Chun-Mei Feng, Anthony Cohn et al.

APS-VLA is a framework for vision-language-action (VLA) token pruning that learns token selection from robot demonstrations while keeping a pretrained VLA backbone frozen. It trains a lightweight Decision Influence Estimator (DIE) using action supervision and an auxiliary action-conditioned visual transition prediction. At deployment, DIE performs one hard token selection/packing step before the frozen policy’s native action generation, enabling faster inference with small task-success changes.

DrivingAI summary
2026-11-11 · arXiv:2611.13182

Making Strides in Symbolic Imitation Learning: Offline Demonstrations, Online Tuning

Bryce MacInnis, Malcolm I. Heywood

The authors propose Symbolic Imitation Learning: distill an oracle policy from an offline dataset into a decision list over if-elif-else rules that deploy linear “motion primitives,” discovered via an EM-based discretization of continuous actions. They then fine-tune thresholds/constants and linear model parameters online using Augmented Random Search (ARS). They report matching oracle performance across 20 RL tasks and exceeding it in 11.

DrivingAI summary
2026-11-11 · arXiv:2611.13203

Audited Gradients, Anchored Policies: Model-Based Reinforcement Learning for Efficient Flow Control

Ruige Kong, Stefania Fresca

The authors address a failure mode in pathwise model-based RL for flow control: a differentiable surrogate can track flow trajectories yet misestimate the action gradient used for policy optimization. They audit surrogate action gradients against high-fidelity central finite differences on held-out states, then fit an extracted low-dimensional feedback law from audited gradients and initialize/anchor a neural policy. Tested on four flow cases, their controllers outperform or match a model-free baseline using 3–156× fewer high-fidelity solver interactions.

DrivingAI summary
2026-11-11 · arXiv:2611.13229

Incorporating World Models in Decision Making for Next Best View Selection in Active Sensing

Nikita Sarawgi, Omey M. Manyar, Satyandra K. Gupta

The authors present a deployment adaptation method for next-best-view (NBV) selection that reuses a fixed action-conditioned world-model predictor but learns a local sensor-return adapter from limited calibration data. The adapter estimates measurement dropout over predicted surfaces and filters candidates accordingly. On a real robot with a structured-light sensor, deployment-aware selection achieves higher measured surface coverage with 60% fewer captures than nominal selection.

DrivingAI summary
2026-11-11 · arXiv:2611.13528

GEMOR-GS: Global Extrinsic-calibration-refined Mapping and Online Relocalization for 3D Gaussian-Splatting SLAM

Yuchen Yang, Ming Ding, Xinyu Wu et al.

GEMOR-GS is a LiDAR–inertial–visual SLAM system that upgrades a 3D Gaussian splatting (3DGS) map into a geometrically accurate, globally refined, and re-enterable world representation. It estimates LiDAR–camera extrinsics offline via direct-method alignment and reuses them. Offline refinement alternates gated differentiable rendering with LiDAR-projection objectives, followed by global reconstruction with online constraints lifted. For tracking loss, it relocalizes via an XFeat bypass map, RANSAC-PnP, and Gaussian RGB-D warp refinement.

DrivingCode availableAI summary
2026-11-11 · arXiv:2611.13819

Game-Enhanced Embodied Learning with Event and Foresight Pretraining

Lianlei Shan, Han Jin, Yaunchao Ma et al.

The authors report event-and-foresight pretraining for InternVLA-A1.5 using 394 human gameplay recordings across 20 games. After 12,400 game-pretraining updates, they compare a transferred checkpoint against the public base using the same suite-specific robot fine-tuning. Across four LIBERO suites, aggregate success increases from 1,718/2,000 (85.90%) to 1,811/2,000 (90.55%) (+4.65 pp), with heterogeneous task outcomes.

DrivingAI summary