Driving · 2026-10-09
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding
Tianhui Cai, Xinglong Sun, Chao Fang, Zhenxin Li, Rui Song, Jose M. Alvarez, Yunxiang Mao, Jiaqi Ma, Langechuan Liu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
AffordDrive3D is an affordance- and geometry-aware world-action model for autonomous driving. It uses a VLM backbone to jointly forecast action-relevant regions (drivable areas and collision-critical regions) and RGB-conditioned future geometry from RGB world-model latents, while an action diffusion model generates future ego trajectories. On NAVSIM, the authors report state-of-the-art 91.3 PDMS and 89.9 EPDMS.
Why it matters
The authors argue that dense geometry alone does not indicate which parts matter for ego motion. By jointly predicting temporally evolving driving affordances together with spatial structure (RGB-conditioned depth), AffordDrive3D aims to align future world-state modeling with planning objectives, improving trajectory planning performance on NAVSIM.
Method
- Builds on a shared causal VLM backbone with learnable affordance, RGB world, geometry, and action tokens; during inference, only the action branch is used to generate the ego trajectory.
- Uses an RGB diffusion world model to predict future visual appearance in latent space, then conditions a geometry (depth) branch on the predicted future RGB latent to recover 3D structure.
- Performs temporal affordance forecasting by decoding affordance token hidden states into drivable-area and collision-critical maps, trained with targets designed for ego reachability and collision risk.
Abstract (from arXiv)
World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle's action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning.
Related papers
- VGGTWorld-VLA: Intent-Conditioned 3D World Evolution for Autonomous Driving
- Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving
- OccWAM: Occupancy World-Action Models with Joint Scene-to-Ego Flow Matching
- PhysWAM: Physically Consistent World Action Model for Autonomous Driving
- DriveWorld3D Evidence Constrained 3D World Modeling and Generative Simulation for Real-World Autonomous Driving
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-08 · Indexed by Pub-AI: 2026-10-09 UTC
AffordDrive3D: Affordance-Aware World-Action Modeling with Spatial Understanding
Preprint posted: 2026-10-08 · Original record ↗
- arxiv
- 2610.11060
Explicit source versions (1)
- v2 · 2026-10-13
- Explicit source version detected · v2
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-10-08 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-action models have recently improved autonomous driving by jointly learning future scene prediction and trajectory generation. Most existing approaches model the future primarily through RGB appearance, and recent works have begun to incorporate geometric prediction to improve spatial understanding. However, dense geometry describes the spatial layout of the entire scene without indicating which parts are most relevant to the ego vehicle's action. For driving, the model must also identify and anticipate where it can safely move and which regions may pose collision risks. Jointly modeling action-relevant regions and future geometry can provide the policy with both driving-relevant cues and their corresponding spatial structure. We therefore propose AffordDrive3D, an affordance- and geometry-aware world-action model that jointly learns future action-relevant regions and spatial structure. In order to capture the scene semantics and driving context needed for driving affordance prediction, we build AffordDrive3D on a VLM backbone to forecast drivable areas and collision-critical regions that directly affect ego motion, while predicting future geometry from RGB world-model latents. On NAVSIM, AffordDrive3D achieves state-of-the-art performance with 91.3 PDMS and 89.9 EPDMS, demonstrating the effectiveness of jointly modeling future affordances and geometry for trajectory planning. Project Page: https://afforddrive3d.github.io/