Latent Dynamics · 2026-10-20
Structured Latent Dynamics for Long-Horizon Embodied World Models
Kevin Chen, Yu-Wei Zhan, Haoran Liang, Sen Cui, Changshui Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors propose HWM, a structured latent dynamics framework that encodes backbone latents into generalized coordinates and momenta, evolves them with dissipative Hamiltonian dynamics, and maps back via a learnable inverse interface. Multi-step transition supervision aligns structured rollouts with latent trajectories from real observations. Experiments show improved higher-order motion prediction, reduced long-horizon energy drift, and better downstream task success when integrated into existing world-model backbones.
Why it matters
Many embodied world models use unconstrained latent transitions that can drift physically when rolled out recursively. By building physical inductive bias directly into the latent transition law (instead of adding physicality post-hoc), HWM aims to improve long-horizon stability and usefulness for planning and control, while remaining compatible with existing perception/decoding interfaces.
Method
- Encode backbone latent z_t into phase-space (q_t, p_t) using an encoder, evolve (q_t, p_t) with a controlled dissipative Hamiltonian system, then map back to latent space for decoding.
- Use dissipative Hamiltonian dynamics with a Hamiltonian energy H_θ(q,p), learned dissipation, and a semi-implicit symplectic Euler update to reduce numerical energy drift.
- Train with multi-objective losses including latent reconstruction and multi-step transition consistency that supervises structured rollout alignment with latent trajectories from real observations.
Abstract (from arXiv)
Reliable long-horizon prediction remains a central challenge for embodied world models. Although recent generative models can synthesize visually plausible action-conditioned futures, their latent dynamics are typically learned through unconstrained transition modules, which may gradually deviate from coherent physical evolution as predictions are recursively rolled forward. We investigate whether explicitly structuring latent state evolution can improve the stability and utility of such imagined trajectories. To this end, we introduce HWM, a structured dynamics framework that represents backbone latents in a learned phase space consisting of generalized coordinates and momenta. Future states are propagated through dissipative Hamiltonian dynamics and mapped back to the original latent representation through a learnable inverse interface. The resulting formulation can be integrated into existing world-model architectures without redesigning their perception or decoding components, while multi-step transition supervision encourages the structured rollout to remain aligned with the latent trajectories induced by real observations. Controlled simulation experiments show that HWM produces more accurate estimates of velocity, acceleration, and jerk and substantially reduces long-horizon energy drift relative to unstructured latent dynamics. We further evaluate the approach across three distinct embodied world-model settings, covering single-arm multi-view prediction with Ctrl-World, dual-arm single-view prediction with EVAC, and task-oriented single-arm prediction with Cosmos-Predict2.5-2B. On RoboCasa, incorporating HWM into Cosmos-Predict2.5-2B yields a 2.7% relative improvement in task success. On real-world AgiBot World trajectories, it improves nDTW by 9.7% with Ctrl-World and Logic by 8.4% with EVAC.
Related papers
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-17 · Indexed by Pub-AI: 2026-10-20 UTC
Structured Latent Dynamics for Long-Horizon Embodied World Models
Preprint posted: 2026-10-17 · Original record ↗
- arxiv
- 2610.24534
Explicit source versions (1)
- v1 · 2026-10-17
- Preprint available
Source date: 2026-10-17 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Reliable long-horizon prediction remains a central challenge for embodied world models. Although recent generative models can synthesize visually plausible action-conditioned futures, their latent dynamics are typically learned through unconstrained transition modules, which may gradually deviate from coherent physical evolution as predictions are recursively rolled forward. We investigate whether explicitly structuring latent state evolution can improve the stability and utility of such imagined trajectories. To this end, we introduce HWM, a structured dynamics framework that represents backbone latents in a learned phase space consisting of generalized coordinates and momenta. Future states are propagated through dissipative Hamiltonian dynamics and mapped back to the original latent representation through a learnable inverse interface. The resulting formulation can be integrated into existing world-model architectures without redesigning their perception or decoding components, while multi-step transition supervision encourages the structured rollout to remain aligned with the latent trajectories induced by real observations. Controlled simulation experiments show that HWM produces more accurate estimates of velocity, acceleration, and jerk and substantially reduces long-horizon energy drift relative to unstructured latent dynamics. We further evaluate the approach across three distinct embodied world-model settings, covering single-arm multi-view prediction with Ctrl-World, dual-arm single-view prediction with EVAC, and task-oriented single-arm prediction with Cosmos-Predict2.5-2B. On RoboCasa, incorporating HWM into Cosmos-Predict2.5-2B yields a 2.7% relative improvement in task success. On real-world AgiBot World trajectories, it improves nDTW by 9.7% with Ctrl-World and Logic by 8.4% with EVAC.