Driving · 2026-10-28
Retrieve to Act: Motion Primitive Graph Retrieval for Robust VLA Execution
Pham Tri Quang, Bao-Ngoc Dao, Minh Duc Nguyen, Duc-Duy Nguyen, Hung T. Ho, Linh Dang Le, Trong-Bao Ho, Thien-Loc Ha, Tan Q. Nguyen, Tuan Dam, Vu N. Duong, Daniel Sonntag, Ngan Le, Khoa D. Doan, Jan Peters, An Thai Le, Duy M. H. Nguyen, Vien Anh Ngo
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Retrieve to Act (R2A) is a test-time scaling method for robust VLA execution under OOD visual shifts. It builds an offline motion primitive graph from successful demonstrations using only kinematic geometry, then uses a query-dependent GNN to retrieve structurally compatible future motion primitives. A learned dimension-wise gating module fuses retrieved priors with frozen VLA actions, improving robustness without finetuning.
Why it matters
The authors address VLA kinematic fragility under out-of-distribution visual changes by bypassing fragile visual features and using geometry-grounded retrieval. R2A leverages successful training experience as a structured motion-primitive graph, localizes execution within this graph during inference, and corrects spatial trajectories while preserving the base policy’s local reactivity—without updating base policy parameters.
Method
- Construct an offline spatiotemporal motion primitive graph where nodes encode kinematic geometry (poses/states/frame-wise changes) and edges capture temporal transitions and cross-task geometric similarity; visual features are discarded.
- At test time, rectifies OOD initial robot states back into the training manifold using a diffusion-based state rectification conditioned on language.
- Retrieve with a query-dependent GNN using a kinematic query, then harmonize retrieved motion with the frozen VLA’s predicted actions via a learned dimension-wise gating mechanism.
Abstract (from arXiv)
Vision-Language-Action (VLA) policies demonstrate remarkable semantic generalization, yet remain highly vulnerable to out-of-distribution (OOD) visual shifts during execution. While existing test-time methods attempt to suppress spurious visual cues, they fundamentally overlook the rich, uncorrupted kinematic priors embedded in the policy's own training experience. We introduce Retrieve to Act (R2A), a test-time scaling framework that formulates OOD robustness purely as a geometric motion primitive retrieval problem. R2A bypasses fragile visual features by structuring successful training demonstrations into an offline spatiotemporal motion primitive graph. During inference, a query-dependent Graph Neural Network dynamically localizes the robot's ongoing execution within this graph, retrieving structurally compatible future motion primitives. These retrieved geometric priors are then seamlessly harmonized with the frozen VLA's predicted actions via a learned, dimension-wise gating mechanism, adaptively rectifying erroneous spatial trajectories while preserving the policy's local reactivity. Extensive experiments across three state-of-the-art VLA architectures, GR00T N1.7, $\pi_{0.5}$, and StableVLA, demonstrate that R2A consistently and significantly enhances robustness against diverse contextual perturbations. R2A achieves absolute success rate improvements of up to +12% in simulation benchmarks (LIBERO-Plus, RoboTwin2.0-Plus) and +40% in real-world deployments, without requiring any finetuning of the base policies. Project page: https://retrieve2act.github.io/r2a/
Related papers
- Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution
- ViDAL: A Visual Dynamics-Grounded Action Latent Space for Vision-Language-Action Models
- Fast-VLA: You Only Think Once for Robot Manipulation
- Causal Analysis of Robot Foundation Models Enables Efficiency Gains
- PaceVLA: Adaptive Temporal Action Quantization in Vision-Language-Action Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-27 · Indexed by Pub-AI: 2026-10-28 UTC
Retrieve to Act: Motion Primitive Graph Retrieval for Robust VLA Execution
Preprint posted: 2026-10-27 · Original record ↗
- arxiv
- 2610.37215
Explicit source versions (1)
- v1 · 2026-10-27
- Preprint available
Source date: 2026-10-27 · First detected: 2026-10-28 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.37215v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-Language-Action (VLA) policies demonstrate remarkable semantic generalization, yet remain highly vulnerable to out-of-distribution (OOD) visual shifts during execution. While existing test-time methods attempt to suppress spurious visual cues, they fundamentally overlook the rich, uncorrupted kinematic priors embedded in the policy's own training experience. We introduce Retrieve to Act (R2A), a test-time scaling framework that formulates OOD robustness purely as a geometric motion primitive retrieval problem. R2A bypasses fragile visual features by structuring successful training demonstrations into an offline spatiotemporal motion primitive graph. During inference, a query-dependent Graph Neural Network dynamically localizes the robot's ongoing execution within this graph, retrieving structurally compatible future motion primitives. These retrieved geometric priors are then seamlessly harmonized with the frozen VLA's predicted actions via a learned, dimension-wise gating mechanism, adaptively rectifying erroneous spatial trajectories while preserving the policy's local reactivity. Extensive experiments across three state-of-the-art VLA architectures, GR00T N1.7, $π_{0.5}$, and StableVLA, demonstrate that R2A consistently and significantly enhances robustness against diverse contextual perturbations. R2A achieves absolute success rate improvements of up to +12% in simulation benchmarks (LIBERO-Plus, RoboTwin2.0-Plus) and +40% in real-world deployments, without requiring any finetuning of the base policies. Project page: https://retrieve2act.github.io/r2a/