Driving · 2026-11-06
VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
Yu-Cheng Chou, Shihao Wang, Jin Huang, Chi-Pin Huang, Chiao-An Yang, Xianzhe Fan, Jing Wang, Zhiqi Li, Yuanguo Kuang, Ruijie Zheng, Johan Bjorck, Sifei Liu, Hang Su, Abhishek Badki, Lei Zhang, Alan Yuille, Linxi "Jim'' Fan, Jan Kautz, Zhiding Yu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
VFlash uses Flash Primitive Decoding to generate complete visual primitives (e.g., boxes and variable-length trajectories) jointly in one forward pass instead of one coordinate at a time. It trains on annotated primitives and accepts structurally valid predictions without requiring target-token agreement for spatial plans, while still using verified speculative decoding for text. Across 2D/3D robot trajectory tasks it reduces prediction errors vs AR decoding, and a VFlash-conditioned world action model improves MolmoSpaces success.
Why it matters
For physical AI, richer visual plans (spatial goals and motion) can be too costly to generate autoregressively under latency constraints. VFlash targets that bottleneck by aligning decoding blocks with the known layout structure of visual primitives and relaxing acceptance to spatial validity rather than exact token sequence matching. This enables faster visual planning with reported gains in spatial error on robot trajectory prediction and improved downstream action success.
Method
- Flash Primitive Decoding (FPD): predict an entire visual primitive jointly using a drafter conditioned on frozen target VLM features, where each prediction block is aligned to the primitive’s type/dimensionality/declared length.
- Relaxed acceptance for visual primitives: VFlash trains on annotated primitives and accepts structurally valid predictions without requiring token-level agreement with the target (schema validator; plus spatial-confidence guard for boxes).
- Verified speculative decoding for text/sub-task plans: VFlash retains target verification for ordinary text, while using direct one-pass primitive drafting for visual planning.
Limitation
The authors report that for boxes, a “spatial-confidence guard” is used for acceptance in the AR fallback; the direct path “accepts every schema-valid box without verification,” and mean F1 drops without fallback (about 20 points on Dense200).
Abstract (from arXiv)
Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
Related papers
- LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
- Retrieve to Act: Motion Primitive Graph Retrieval for Robust VLA Execution
- PDRVLA: A Vision-Language-Action Model with Physical Dynamics Representations
- Dream What I Can: Sparse Visual Planning for Long-Horizon Robot Manipulation
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-05 · Indexed by Pub-AI: 2026-11-06 UTC
VFlash: Fast, Accurate Visual Planning via Flash Primitive Decoding
Preprint posted: 2026-11-05 · Original record ↗
- arxiv
- 2611.08449
Explicit source versions (1)
- v1 · 2026-11-05
- Preprint available
Source date: 2026-11-05 · First detected: 2026-11-06 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.08449v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Models for physical AI tasks, such as robotics and autonomous driving, need not reason solely in language. They can use visual plans--points, bounding boxes, and trajectories--to represent spatial goals and anticipate motion. However, autoregressive vision-language models generate these plans one coordinate token at a time, making richer predictions increasingly costly. We introduce VFlash, a method for fast, accurate visual planning through Flash Primitive Decoding. VFlash exploits the known structure of visual primitives to predict their coordinates jointly, using a lightweight drafter that generates an entire primitive in one forward pass. The layout adapts to the primitive's type, dimensionality, and length, supporting compact boxes and long, variable-length trajectories. Crucially, spatial accuracy need not require reproducing an autoregressive model's exact token sequence. VFlash therefore trains on annotated primitives and directly accepts structurally valid predictions without target-token agreement. Across the evaluated 2D and 3D robot trajectory tasks, VFlash consistently reduces prediction errors relative to autoregressive decoding. Preliminary single-rollout measurements indicate approximately 25x faster generation for 3D point motion forecasting. A world action model conditioned on VFlash plans achieves 70.2% average success across four MolmoSpaces task suites, compared with 62.6% for the baseline.
