Robotics · 2026-10-16
Causal Analysis of Robot Foundation Models Enables Efficiency Gains
Aiden Swann, Mathilde Papillon, Mozes Jacobs, Siddharth Boppana, Tal Haklay, Eric Bigelow, Andrew Lee, Conor Watts, Thomas Icard, Mac Schwager, Owen Lewis, Jack Merullo, Ekdeep Singh Lubana, Atticus Geiger, Thomas Fel, Matthew Kowal
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors analyze, with activation patching, how three vision-language-action (VLA) robot foundation models route image and instruction information into an action expert. They find a narrow middle-to-late “action-conditioning interface” (1–3 cross-attention layers) and use it to sparsify action-expert conditioning. For MolmoBot, restricting to 3/36 layers reduces inference FLOPs by up to 33% while keeping comparable task success; they also report real-robot transfers for π0.5.
Why it matters
Mechanistic identification of where action-relevant information enters the action expert enables targeted architecture changes rather than heuristic pruning. The reported interface-based simplification yields compute/latency reductions while aiming to preserve closed-loop manipulation performance across simulation and (for π0.5) real-robot deployment.
Method
- Activation patching and “interface patching” replace conditioning to selected action-expert blocks with donor values while keeping the VLM backbone computation fixed, measuring behavioral transfer.
- Across π0.5, MolmoAct2, and MolmoBot, the authors locate a narrow band of mid-to-late cross-attention layers that causally shapes action generation (the action-conditioning interface).
- They restrict action-expert conditioning to the identified layers (e.g., keep only layers 23–25) and evaluate closed-loop success plus compute/latency impacts.
Limitation
The paper states that for π0.5 the efficiency gain is much smaller than for MolmoBot, and notes the ablation removes a relatively minor amount of FLOPs because the action expert reads prefix keys and values already computed by the VLM.
Abstract (from arXiv)
Vision-language-action (VLA) models are a promising approach to general purpose robot manipulation. While the internal computation of vision-language models, which compose the backbone of VLAs, has been studied extensively, less is known about how their perceptual and semantic representations are transformed into action in VLA models. We study this question mechanistically in three frontier VLA models ($\pi_{0.5}$, MolmoAct2, and MolmoBot) by analyzing the causal pathways through which the input image and the instruction shape actions. Across the three models, activation patching shows that causally relevant information influences the action expert (AE) through a narrow band of mid-to-late cross-attention layers, which we term the action-conditioning interface. The content of this information changes with token and layer within the model. Early-to-mid layer representations of the target noun token determine which object is selected, whereas mid-to-late layers carry the spatial information in text tokens that redirects the movement when patched. Where spatial information is stored in prompt tokens various based on the architecture. In MolmoAct2 the model forms task-conditioned representations at the text positions that follow the instruction, whereas $\pi_{0.5}$ additionally stores this information at image token positions. We show that these findings can be translated directly into improving model efficiency. We find that restricting MolmoBot's action expert conditioning to only 3 out of 36 layers reduces inference FLOPs by up to $33\%$ while maintaining comparable task success with the default model across 3,000 evaluation episodes ($46.7\%$ vs. $46.9\%$), and the same principle carries over to real-robot deployment of $\pi_{0.5}$.
Related papers
- Fast-VLA: You Only Think Once for Robot Manipulation
- MIKASA-Robo-VLA: Benchmarking Memory in VLA Models for Long-Horizon Manipulation
- PaceVLA: Adaptive Temporal Action Quantization in Vision-Language-Action Models
- Axiom-0: Hierarchical Self-Refinement for Vision-Language-Action Policies
- Robota-Embed: Unified VLA Embeddings for Robot In-Context Learning and Evolution
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-14 · Indexed by Pub-AI: 2026-10-16 UTC
Causal Analysis of Robot Foundation Models Enables Efficiency Gains
Preprint posted: 2026-10-14 · Original record ↗
- arxiv
- 2610.20750
Explicit source versions (1)
- v1 · 2026-10-14
- Preprint available
Source date: 2026-10-14 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Vision-language-action (VLA) models are a promising approach to general purpose robot manipulation. While the internal computation of vision-language models, which compose the backbone of VLAs, has been studied extensively, less is known about how their perceptual and semantic representations are transformed into action in VLA models. We study this question mechanistically in three frontier VLA models ($π_{0.5}$, MolmoAct2, and MolmoBot) by analyzing the causal pathways through which the input image and the instruction shape actions. Across the three models, activation patching shows that causally relevant information influences the action expert (AE) through a narrow band of mid-to-late cross-attention layers, which we term the action-conditioning interface. The content of this information changes with token and layer within the model. Early-to-mid layer representations of the target noun token determine which object is selected, whereas mid-to-late layers carry the spatial information in text tokens that redirects the movement when patched. Where spatial information is stored in prompt tokens various based on the architecture. In MolmoAct2 the model forms task-conditioned representations at the text positions that follow the instruction, whereas $π_{0.5}$ additionally stores this information at image token positions. We show that these findings can be translated directly into improving model efficiency. We find that restricting MolmoBot's action expert conditioning to only 3 out of 36 layers reduces inference FLOPs by up to $33\%$ while maintaining comparable task success with the default model across 3,000 evaluation episodes ($46.7\%$ vs. $46.9\%$), and the same principle carries over to real-robot deployment of $π_{0.5}$.