Robotics · 2026-11-03
ODE Policy: Robot Learning via Action Dynamics, Not Actions
Jindou Jia, Jingliang Li, Kuangji Zuo, Tuo An, Yuxuan Hu, Mengfei Zhao, Bofan Lyu, Jiaqi Bai, Daoyuan Zhu, Hengyu Dai, Junpeng Zhang, Jianfei Yang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
ODE policy learns the dynamics of robot actions as an ODE and integrates a learned action vector field over physical time, so executed trajectories are continuous by construction. Trained by matching integrated actions to targets in IL (over action chunks) and in RL (single control steps), with a diversity head for multi-modal behavior. It is evaluated across IL (including VLA and world action model heads) and model-based RL on multiple real and simulated robot tasks.
Why it matters
The authors propose a shift from discrete, direct action prediction to continuous-time action evolution: integrating the learned vector field ties consecutive actions together, addressing action discontinuities and oscillations attributed to direct-action policies. They report strong results across visuomotor IL, VLA/WAM action-head replacements, and an RL actor replacement, including smoother motions and faster action generation.
Method
- Parameterize action dynamics as a neural ODE vector field f_θ(a(t), s_t, z_t) and integrate it over physical time so each action is obtained from the previous one.
- In IL, integrate over short action chunks and supervise by matching the integrated action path without regressing velocities; add contractive perturbation for robustness.
- Add an optional diversity head that randomizes the direction of the field; in RL, integrate over single control steps and use the same field as the actor in model-based RL.
Abstract (from arXiv)
Most robot policies, whether trained by imitation learning (IL) or reinforcement learning (RL), map observations to action values at discrete control steps and leave the continuity of physical motion to be learned implicitly. We propose ODE policy, which learns the dynamics of the action rather than the action itself. A neural ordinary differential equation (ODE) learns the vector field of the action by matching the integrated actions to the targets over short action chunks in IL and single control steps in RL. A lightweight diversity head captures multi-modal behavior by randomizing the direction of the field. At deployment, each action is integrated from the previous one, so the executed trajectory is continuous by construction. We evaluate it as a visuomotor policy, as the action head of vision-language-action (VLA) and world action models (WAMs), and as the actor of model-based RL against 14 baselines on unimanual, bimanual, dexterous, locomotion, and loco-manipulation tasks. In IL, against flow matching, it reaches 5.9$\times$ the success, 5.4$\times$ the generalization, and 3.9$\times$ the mode balance after one epoch, and generates actions 10.8$\times$ faster. It mostly moves more smoothly than human demonstrations. On 14 real-world tasks across single-arm, dual-arm, dexterous, and humanoid robots, it achieves the highest success rate. In RL, it further speeds up SuperMimic for humanoid motion tracking and smooths its actions. Efficient in learning, data, parameters, and execution, ODE policy offers a promising path to scaling robot learning.
Related papers
- Residual Modeling Closes the Regression and Generative Policy Gap in Robot Learning
- What Should Robot Policies Learn to Predict? Rethinking Supervision Targets in Vision-Language-Action Learning
- DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
- $\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation
- Forward Dynamics Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
ODE Policy: Robot Learning via Action Dynamics, Not Actions
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.01729
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.01729v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Most robot policies, whether trained by imitation learning (IL) or reinforcement learning (RL), map observations to action values at discrete control steps and leave the continuity of physical motion to be learned implicitly. We propose ODE policy, which learns the dynamics of the action rather than the action itself. A neural ordinary differential equation (ODE) learns the vector field of the action by matching the integrated actions to the targets over short action chunks in IL and single control steps in RL. A lightweight diversity head captures multi-modal behavior by randomizing the direction of the field. At deployment, each action is integrated from the previous one, so the executed trajectory is continuous by construction. We evaluate it as a visuomotor policy, as the action head of vision-language-action (VLA) and world action models (WAMs), and as the actor of model-based RL against 14 baselines on unimanual, bimanual, dexterous, locomotion, and loco-manipulation tasks. In IL, against flow matching, it reaches 5.9$\times$ the success, 5.4$\times$ the generalization, and 3.9$\times$ the mode balance after one epoch, and generates actions 10.8$\times$ faster. It mostly moves more smoothly than human demonstrations. On 14 real-world tasks across single-arm, dual-arm, dexterous, and humanoid robots, it achieves the highest success rate. In RL, it further speeds up SuperMimic for humanoid motion tracking and smooths its actions. Efficient in learning, data, parameters, and execution, ODE policy offers a promising path to scaling robot learning.
