Driving · 2026-10-08

Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving

Xingtai Gui, Yucheng Zhou, Dongqian Guo, Jiahao Gong, Feiyang Tan, Jianbing Shen

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

GeoCoTDrive is an explicit geometric chain-of-thought framework for vision-language-action autonomous driving. It “think[s] with 2D first,” by grounding planning-relevant 2D regions, then “drive[s] with dedicated 3D priors” by retrieving localized geometry tokens from a frozen geometric foundation model within those regions and interleaving them into the autoregressive context for trajectory generation. It introduces planning-relevant grounding and the PlanningGrounding dataset (146K annotations).

Why it matters

The authors target a stated mismatch in VLA driving: actions require precise 3D geometric cues, while VLM/VLA reasoning is largely semantic and 2D. GeoCoTDrive explicitly grounds planning-critical regions and retrieves localized 3D priors conditioned on that grounding, aiming to improve safety-critical planning. The PlanningGrounding benchmark supplies region-level, planning-relevant grounding supervision for this coupling.

Method

Generate an explicit grounding trace: autoregressively produce planning-relevant 2D bounding boxes corresponding to local decision-critical cues.
Retrieve localized geometric priors: sample region-level tokens from a frozen geometric foundation model (Depth Anything V3 and/or VGGT) inside the grounded boxes, then align and concatenate geometry tokens.
Interleave geometry tokens into the autoregressive context to condition trajectory generation; train with teacher forcing using supervised grounding and expert trajectories.

Limitation

GeoCoTDrive primarily relies on front-view images without explicit multi-view or temporal modeling, limiting its ability to account for side and rear traffic, prolonged occlusions, and interaction histories. Localized geometry retrieval also depends on grounding accuracy.

Abstract (from arXiv)

Vision-language-action~(VLA) models have emerged as a promising paradigm for autonomous driving. However, existing VLA models still suffer from a fundamental mismatch: driving actions require precise 3D geometric cues, while visual-language understanding and reasoning are largely conducted in a 2D semantic space. In this paper, we propose GeoCoTDrive, an explicit geometric chain-of-thought framework that grounds geometry in a planning-oriented manner. GeoCoTDrive follows a think with 2D first, drive with dedicated 3D priors paradigm. It first grounds 2D regions corresponding to decision-critical cues, and then retrieves localized 3D priors by sampling features from a geometric foundation model within the grounded regions. These localized geometric features are interleaved into the autoregressive context to support the trajectory generation. To supervise this process, we introduce planning-relevant grounding, a new region-level grounding task that focuses on local spatial cues directly affecting ego planning decisions, and construct the PlanningGrounding dataset to endow VLAs with planning-oriented grounding capability. Experiments across multiple end-to-end autonomous driving benchmarks show that GeoCoTDrive consistently improves safety-critical planning performance, demonstrating the effectiveness of the explicit geometric chain-of-thought process for VLA-based planning.

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-10-07 · Indexed by Pub-AI: 2026-10-08 UTC

preprint