Driving · 2026-10-20
Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving
Jie Wang, Guang Li, Zhijian Huang, Jinlong Li, Chenxu Dang, Hangjun Ye, Yahong Han, Long Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose LECDrive under a “Think Densely, Act Sparsely” paradigm for vision-language-action autonomous driving. LECDrive injects compact task-specific latent tokens into a VLM (Qwen2.5-VL-7B) as a Latent Expert Cognitive Chain with three stages: DINOv3 (visual), SAM3 (semantic), and DepthAnything3 (geometry). Dense world modeling is trained via dedicated dense perception heads, while actions remain sparse.
Why it matters
The paper targets dense visual information loss caused by training VLA models with sparse supervision (language instructions or trajectory signals). The authors argue that adding explicit latent mechanisms that encourage dense world representations—progressing from visual to semantic to geometric knowledge—can improve scene understanding and trajectory planning performance while keeping efficient inference.
Method
- Injects compact task-specific latent tokens (<DINO_TOKEN>, <SAM_TOKEN>, <DEPTH_TOKEN>) after visual tokens for each view in Qwen2.5-VL-7B, forming a progressive latent expert cognitive chain.
- Uses an interdependent curriculum where earlier supervision refines later representations: DINOv3 for feature-level visualization, SAM3 for traffic-relevant semantics, and DepthAnything3 for spatial geometry.
- Adds three lightweight dense perception heads (representation, semantic segmentation, and depth) trained with pseudo-labels from frozen expert models; trajectory planning combines discrete motion primitives with continuous offset refinement.
Limitation
The authors do not provide an explicit limitation in the provided text.
Abstract (from arXiv)
Autonomous driving fundamentally demands dense understanding of visual, semantic, and geometric information, yet existing Vision-Language-Action (VLA) models are typically trained with sparse supervision such as language instructions or trajectory signals. This direct mapping from dense visual observations to sparse representations forces the model to compress or even discard critical scene information essential for safe driving. We argue that this limitation does not stem from insufficient model capacity, but from the lack of explicit mechanisms that encourage the formation of dense world representations in the latent space. In this spirit, we propose a novel paradigm for driving intelligence, Think Densely, Act Sparsely, and instantiate it with an efficient and interpretable framework, LECDrive (Latent Expert Cognitive Chains for Vision-Language-Action autonomous Driving). The core idea is to mimic the hierarchical perception process of humans by embedding a progressive chain of latent expert cognition within the VLA model. Concretely, we inject a compact set of task-specific latent tokens as carriers, enabling the model to internalize dense knowledge from foundational vision models, including visual representations (DINOv3), semantic structures (SAM3), and spatial geometry (DepthAnything3), while maintaining efficient inference. This interdependent cognitive chain follows a curriculum learning principle, where supervision is progressively structured from representation to semantics and geometry. Extensive experiments demonstrate that LECDrive consistently improves the performance of base VLMs across multiple autonomous driving benchmarks, covering key tasks such as object perception, state prediction, and trajectory planning. These findings suggest that constructing dense world models through structured latent expert chains is a promising direction for VLA-based autonomous driving.
Related papers
- Explicit Geometric Chain-of-Thought for Vision-Language-Action in Autonomous Driving
- DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving
- MORE: Learning Motion-Oriented Registers for Driving World Models and Planning
- Language2Drive: Do Vision-Language-Action Models Really Follow the Instructions in Autonomous Driving?
- Scene-Adaptive VLA: Efficient Autonomous Driving via Dynamic Layer Routing
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-19 · Indexed by Pub-AI: 2026-10-20 UTC
Think Densely, Act Sparsely: Latent Expert Cognitive Chains for Vision-Language-Action Autonomous Driving
Preprint posted: 2026-10-19 · Original record ↗
- arxiv
- 2610.25734
Explicit source versions (1)
- v1 · 2026-10-19
- Preprint available
Source date: 2026-10-19 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Autonomous driving fundamentally demands dense understanding of visual, semantic, and geometric information, yet existing Vision-Language-Action (VLA) models are typically trained with sparse supervision such as language instructions or trajectory signals. This direct mapping from dense visual observations to sparse representations forces the model to compress or even discard critical scene information essential for safe driving. We argue that this limitation does not stem from insufficient model capacity, but from the lack of explicit mechanisms that encourage the formation of dense world representations in the latent space. In this spirit, we propose a novel paradigm for driving intelligence, Think Densely, Act Sparsely, and instantiate it with an efficient and interpretable framework, LECDrive (Latent Expert Cognitive Chains for Vision-Language-Action autonomous Driving). The core idea is to mimic the hierarchical perception process of humans by embedding a progressive chain of latent expert cognition within the VLA model. Concretely, we inject a compact set of task-specific latent tokens as carriers, enabling the model to internalize dense knowledge from foundational vision models, including visual representations (DINOv3), semantic structures (SAM3), and spatial geometry (DepthAnything3), while maintaining efficient inference. This interdependent cognitive chain follows a curriculum learning principle, where supervision is progressively structured from representation to semantics and geometry. Extensive experiments demonstrate that LECDrive consistently improves the performance of base VLMs across multiple autonomous driving benchmarks, covering key tasks such as object perception, state prediction, and trajectory planning. These findings suggest that constructing dense world models through structured latent expert chains is a promising direction for VLA-based autonomous driving.