Driving · 2026-11-02
Cross-Embodiment Motion Planning by Coupling Body-Agnostic World Model with Differentiable Ego-Dynamics
Zhidong Wang, Zirui Li, Haoran Xu, Yujie Yan, Zitong Shan, Jingsong Liang, Han Yu, Chen Lv
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
Hourglass factorizes cross-embodiment local motion planning at the camera ego-motion interface: a body-agnostic recurrent world model conditioned only on ego-motion and camera parameters, coupled to each robot’s differentiable ego-dynamics (a reduced-order template) only through the implied camera ego-motion. The authors learn a generalist parameter-conditioned policy once via coupled latent imagination, and identify template parameters on new robots for deployment.
Why it matters
The authors address a core limitation of cross-embodiment planners: embodiment changes both feasible free space and reachable commands, yet prior world-model approaches entangle environment and embodiment (command-conditioned) or omit the command-to-motion mapping (pose-conditioned). By conditioning the world model only on ego-motion, the policy can become body-aware through differentiable ego-dynamics while keeping the world model reusable across robots and camera mounts, enabling zero-shot transfer to unseen embodiments.
Method
- Train a recurrent state-space world model conditioned only on camera ego-motion Δξ and camera parameters c, excluding commands and embodiment parameters from the model (body-agnostic by construction).
- Use a differentiable reduced-order ego-dynamics template fθ with identified physical parameters that maps commanded velocities to camera ego-motion; couple template and world model only through Δξ during imagination.
- Learn a generalist parameter-conditioned policy once by analytic gradients through coupled latent imagination, and deploy on new robots by identifying θ from brief teleoperated driving.
Limitation
The main limitations are the latent clearance precision near obstacles, the body sizes covered in training, and evaluation in a single simulated building.
Abstract (from arXiv)
Motion planning in clutter is embodiment-specific: the feasible free space and reachable commands depend on each robot's geometry and dynamics. Existing cross-embodiment policies output normalized commands that disregard these differences, or compensate through per-robot training and distillation. World models can in principle pool cross-embodiment data, yet command-conditioned models bind their latent dynamics to a single action space, while pose-conditioned models omit the mapping from commands to body-specific motion. On flat ground, however, embodiment affects the planner's observations only through the camera ego-motion. This letter proposes Hourglass, an hourglass-shaped architecture that factorizes cross-embodiment motion planning at this interface: a body-agnostic world model and each robot's differentiable ego-dynamics, which map commands to that ego-motion, are coupled only through it. The world model is trained on data pooled across embodiments and camera mounts; each robot's ego-dynamics is a reduced-order template with identified physical parameters. A generalist parameter-conditioned policy is learned once at this junction: each imagined step rolls out the template and then the world model, and imagination training through each robot's template makes the policy body-aware. On seven simulated robots of five locomotion types, the generalist policy outperforms imitation-learning and diffusion-policy baselines on the same world model by 3.8--10.6 percentage points in success rate and matches per-robot specialists without dedicated training, while approaching its parameter-conditioned MPPI teacher within 1.4--2.8 points without an online map. After three minutes of teleoperated driving for identification, the synthetic-data-trained model transfers zero-shot to a real skid-steer robot unseen in training, navigating unseen indoor scenes.
Related papers
- Cross-Embodiment Robot Foundation World Models with Latent Actions
- EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming
- Continuous-Action World Models: Learning Cross-Embodiment Spatiotemporal Dynamics Priors
- Ropedia W0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience
- RynnWorld-Latent: A Cross-Embodiment World Model Conditioned on Universal Latent Actions
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-02 · Indexed by Pub-AI: 2026-11-02 UTC
Cross-Embodiment Motion Planning by Coupling Body-Agnostic World Model with Differentiable Ego-Dynamics
First announced by arXiv: 2026-11-02 · Original record ↗
- arxiv
- 2610.41330
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-02 · First detected: 2026-11-02 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2610.41330v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Motion planning in clutter is embodiment-specific: the feasible free space and reachable commands depend on each robot's geometry and dynamics. Existing cross-embodiment policies output normalized commands that disregard these differences, or compensate through per-robot training and distillation. World models can in principle pool cross-embodiment data, yet command-conditioned models bind their latent dynamics to a single action space, while pose-conditioned models omit the mapping from commands to body-specific motion. On flat ground, however, embodiment affects the planner's observations only through the camera ego-motion. This letter proposes Hourglass, an hourglass-shaped architecture that factorizes cross-embodiment motion planning at this interface: a body-agnostic world model and each robot's differentiable ego-dynamics, which map commands to that ego-motion, are coupled only through it. The world model is trained on data pooled across embodiments and camera mounts; each robot's ego-dynamics is a reduced-order template with identified physical parameters. A generalist parameter-conditioned policy is learned once at this junction: each imagined step rolls out the template and then the world model, and imagination training through each robot's template makes the policy body-aware. On seven simulated robots of five locomotion types, the generalist policy outperforms imitation-learning and diffusion-policy baselines on the same world model by 3.8--10.6 percentage points in success rate and matches per-robot specialists without dedicated training, while approaching its parameter-conditioned MPPI teacher within 1.4--2.8 points without an online map. After three minutes of teleoperated driving for identification, the synthetic-data-trained model transfers zero-shot to a real skid-steer robot unseen in training, navigating unseen indoor scenes.
