Driving · 2026-10-23
Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
Yunfei Xie, Howard H. Qian, Shiyi Lan, Kaiyu Hang, Chen Wei
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose Sensorimotor Alignment (S²), a training-free offline metric that measures overlap between nearest-neighbor neighborhoods in a model’s sensory representation (image + text features) and motor representation (end-effector trajectories). They report positive correlations between S² and downstream robot task success across embodied model families, and show S² is more predictive than VQA scores or video-quality metrics.
Why it matters
Closed-loop robot evaluations are expensive. The authors argue S² can prioritize embodied models offline by quantifying whether sensory representations are organized for action—by aligning sensory neighborhood structure with motor-trajectory neighborhood structure. They further use S² to provide representation-level explanations for behaviors observed in vision-language-action models and world action models.
Method
- Define sensory representation from robot foundation model image frame + task instruction (image and text encoders; average-pooled token features).
- Define motor representation from recorded end-effector trajectories (positions, orientations, gripper state over next h steps) using distances reflecting translation/rotation geometry and DTW for speed differences.
- Compute Sensorimotor Alignment (S²) as mutual nearest-neighbor overlap between k-NN sets in sensory space and motor space; greater overlap indicates stronger alignment.
Limitation
For WAMs, only 3 of the 6 models are compared with video-quality metrics because the others do not generate future video during closed-loop task evaluation.
Abstract (from arXiv)
Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.
Related papers
- Language-Forcing: Advancing Humanoid Loco-Manipulation via Semantic Alignment of Whole-Body World Action Models
- UniWALM: Unified World Action Language Model for Embodied Robot Control
- Causal Analysis of Robot Foundation Models Enables Efficiency Gains
- Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
- EmbodiedRSI: Active Continual Robot Learning Through Hypothesis-Guided Co-Evolution
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-22 · Indexed by Pub-AI: 2026-10-23 UTC
Sensorimotor Alignment: A Platonic Proxy for Embodied Policy Success
Preprint posted: 2026-10-22 · Original record ↗
- arxiv
- 2610.31670
Explicit source versions (1)
- v1 · 2026-10-22
- Preprint available
Source date: 2026-10-22 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Robotic foundation models are typically evaluated through costly closed-loop rollouts, which motivates cheap offline proxies for model capability. The Platonic Representation Hypothesis (PRH) suggests that as representations improve, different modalities such as vision and language converge toward a shared statistical model of reality. We ask whether motion trajectories form another such modality: as embodied models become more capable, should their sensory representations converge toward the structure of physical motion? To examine this, we propose Sensorimotor Alignment ($S^2$), a training-free offline metric that measures how well a model's sensory representation aligns with the geometry of physical motion. We find that $S^2$ correlates positively with downstream robot task success across embodied model families, making it a low-cost offline proxy for prioritizing models before closed-loop evaluation. $S^2$ can also identify promising vision-language model (VLM) backbones even before costly fine-tuning into vision-language-action models (VLAs). Beyond this correlation, $S^2$ offers representation-level explanations for phenomena observed in VLAs and world action models (WAMs). With $S^2$, we extend PRH to embodied models and make sensorimotor alignment empirically measurable, offering insight into why an embodied model succeeds beyond its closed-loop success rate.