Driving · 2026-11-10

DriveFoundry: Co-Evolving Data and Model for Efficient Closed-Loop Iteration in Autonomous Driving

Ziyang Yao, Yuncheng Jiang, Fang Li, Haochen Liu, Shaoqing Xu, Wen Zhang, Xiangyang Chu, Zibin Guo, Jiale Hou, Chuang Wan, Jia Hua, Gengzhao Xiang, Xiaoran Luo, Chong He, Yuechen Luo, Junzhe Wang, Zhiyu Sun, Jun Zhang, Wei Li, Lei Li, Hongwei Xie, Bing Wang, Yuannan Shen, Jianwei Cui, Guang Chen, Hangjun Ye, Jingwei Zhao

arXiv:2611.11532PDF

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors introduce DriveFoundry, a closed-loop driving framework that couples structured causal data refinement with adaptive inference to improve iteration efficiency under data and computation budgets. It uses a Data Engine for candidate scene refinement and verified failure-guided VLM-assisted annotation, and a unified vision-language model (Xiaomi-Drive-o1) with block-wise latent refill and a value-of-computation head to allocate computation per decision.

Why it matters

The paper targets learning reusable decision knowledge from rare critical events for autonomous driving while respecting strict runtime constraints. DriveFoundry’s loop aims to make data and annotation more efficient, and its adaptive inference aims to meet compute limits by selecting tasks and allocating latent tokens based on expected benefit and cost.

Method

  • Data Engine: builds traceable scene-candidate records linking scene evidence and factor-decision relations to candidate trajectories/outcome assessments, using candidate comparisons and counterfactual annotations; verified failures guide VLM-assisted annotation and expert correction.
  • Model engine (Xiaomi-Drive-o1): unifies Perception, Causation, Action, World, and Value within a shared vision-language backbone; uses block-wise latent refill passes candidate-specific evidence and predicted visual consequences between task blocks.
  • Adaptive inference: a learned value-of-computation head selects tasks and allocates latent tokens based on expected benefit and cost, enabling compute-adaptive reasoning; authors report token savings while matching full reasoning performance.
Abstract (from arXiv)

A driving foundation model must learn reusable decision knowledge from rare critical events and apply it under strict runtime constraints. We introduce DriveFoundry, a closed-loop framework improving iteration efficiency under data and computation budgets by coupling structured causal data refinement with adaptive inference. The Data Engine builds traceable scene-candidate records linking scene evidence and factor-decision relations to candidate trajectories and outcome assessments. Candidate comparisons and counterfactual annotations clarify how scene factors affect behavioral feasibility and preference. Verified model failures guide VLM-assisted annotation and expert correction, refining training data and annotation systems. The model engine, Xiaomi-Drive-o1, unifies Perception, Causation, Action, World, and Value within a shared vision-language backbone to enable cross-task knowledge transfer. Block-wise latent refill passes candidate-specific evidence and predicted visual consequences between task blocks, connecting scene understanding, planning, and evaluation. A learned value-of-computation head selects tasks and allocates latent tokens based on expected benefit and cost, adapting inference to each decision. On PhysicalAI, the 9B model achieves the lowest ADE@1 across evaluated splits and horizons among tested models. In AlpaSim, it outperforms Alpamayo 1.5 across all metrics and achieves lower close-encounter and off-road rates than the 34B Alpamayo 2 Super, while trailing in progress and AlpaSim scores. For the 0.8B variant, joint training reduces 5 s minADE@6 by about 54% without altering the inference path or token budget. Adaptive inference matches full reasoning performance using only 28% of tokens. Real-vehicle tests confirm runtime feasibility, supporting a unified paradigm for learning from critical experience and dynamically allocating computation.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First announced by arXiv: 2026-11-10 · Indexed by Pub-AI: 2026-11-10 UTC

preprint

DriveFoundry: Co-Evolving Data and Model for Efficient Closed-Loop Iteration in Autonomous Driving

Ziyang Yao, Yuncheng Jiang, Fang Li, Haochen Liu, Shaoqing Xu, Wen Zhang, Xiangyang Chu, Zibin Guo, Jiale Hou, Chuang Wan, Jia Hua, Gengzhao Xiang, Xiaoran Luo, Chong He, Yuechen Luo, Junzhe Wang, Zhiyu Sun, Jun Zhang, Wei Li, Lei Li, Hongwei Xie, Bing Wang, Yuannan Shen, Jianwei Cui, Guang Chen, Hangjun Ye, Jingwei Zhao

First announced by arXiv: 2026-11-10 · Original record ↗

arxiv
2611.11532
Explicit source versions (1)
  • v1 · Date not supplied
  1. Preprint available

    Source date: 2026-11-10 · First detected: 2026-11-10 UTC · Historical baseline

    Source evidence (1)
    • Official evidence ↗ · RSS guid=oai:arXiv.org:2611.11532v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv

A driving foundation model must learn reusable decision knowledge from rare critical events and apply it under strict runtime constraints. We introduce DriveFoundry, a closed-loop framework improving iteration efficiency under data and computation budgets by coupling structured causal data refinement with adaptive inference. The Data Engine builds traceable scene-candidate records linking scene evidence and factor-decision relations to candidate trajectories and outcome assessments. Candidate comparisons and counterfactual annotations clarify how scene factors affect behavioral feasibility and preference. Verified model failures guide VLM-assisted annotation and expert correction, refining training data and annotation systems. The model engine, Xiaomi-Drive-o1, unifies Perception, Causation, Action, World, and Value within a shared vision-language backbone to enable cross-task knowledge transfer. Block-wise latent refill passes candidate-specific evidence and predicted visual consequences between task blocks, connecting scene understanding, planning, and evaluation. A learned value-of-computation head selects tasks and allocates latent tokens based on expected benefit and cost, adapting inference to each decision. On PhysicalAI, the 9B model achieves the lowest ADE@1 across evaluated splits and horizons among tested models. In AlpaSim, it outperforms Alpamayo 1.5 across all metrics and achieves lower close-encounter and off-road rates than the 34B Alpamayo 2 Super, while trailing in progress and AlpaSim scores. For the 0.8B variant, joint training reduces 5 s minADE@6 by about 54% without altering the inference path or token budget. Adaptive inference matches full reasoning performance using only 28% of tokens. Real-vehicle tests confirm runtime feasibility, supporting a unified paradigm for learning from critical experience and dynamically allocating computation.