Latent Dynamics · 2026-10-14
LeHami: Symplectic Latent Dynamics for World Models
Romina Yalovetzky, Randall Balestriero, Yann LeCun
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
LeHami is a JEPA-style latent world model where the predictor is replaced by an action-conditioned Hamiltonian flow on a structured latent phase space (q,p), integrated with an exactly symplectic leapfrog step. The authors prove the rollout map is exactly symplectic and preserves phase-space volume at every horizon, ruling out uniform contraction, asymptotically stable fixed points, and merging of latent states. On visual pendulums, it improves long-horizon velocity prediction and maintains constant latent volume where an unconstrained JEPA contracts.
Why it matters
The authors address a key failure mode in autoregressive latent predictors: contraction that compounds over long rollouts. By imposing symplectic structure on the inference-time transition, LeHami provides stepwise geometric guarantees that are independent of learned parameters and action sequence. The model also supports continuous-time sub-frame integration and time reversal, and it aims to represent the full mechanical state (position and momentum) for prediction and control.
Method
- Train a JEPA encoder to output a position-like latent q, and obtain a momentum-like latent p as the time derivative of q via a fixed Savitzky–Golay stencil (no learned parameters in the differentiation).
- Replace the unconstrained predictor with an action-conditioned separable Hamiltonian Hφ(q,p,u)=Kφ(p)+Vφ(q,u), and advance (q,p) using an exactly symplectic leapfrog step.
- Use multi-step self-fed rollout training with an MSE loss over predicted q and p across a K-step horizon, plus an embedding regularizer applied to ground-truth embeddings.
Limitation
Shape, not size: volume preservation forbids degenerate failures but does not bound the growth of rollout error.
Abstract (from arXiv)
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltonian flow over a structured latent state (q, p) integrated using an exactly symplectic leapfrog step. This construction preserves phase-space volume at every rollout step and rules out uniform contraction, asymptotically stable fixed points, and merging of distinct latent states, independently of the learned parameters or action sequence. We obtain the momentum-like component from temporal derivatives of the learned position-like representation, providing an explicit second-order state for prediction and control. On conservative and actuated visual pendulum environments, LeHami substantially improves long-horizon velocity prediction over an unconstrained JEPA world model and empirically maintains constant latent volume where the baseline contracts. The learned flow also supports accurate integration at unseen sub-frame step sizes and numerical time reversal. These results show how geometric structure can provide inference-time guarantees that are complementary to representation regularization in latent world models.
Related papers
- Structured Latent Dynamics for Long-Horizon Embodied World Models
- SplitJEPA: Learning Invariant and Variant Latent Worlds without Reconstruction
- Hamiltonian JEPA: Action-Conditioned World Models with an Inherited Control State
- Enhancing Latent World Models with Associative Memory
- Bilinear World Models: Learning Representations with Structured Dynamics for Efficient Control
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-11 · Indexed by Pub-AI: 2026-10-14 UTC
LeHami: Symplectic Latent Dynamics for World Models
Preprint posted: 2026-10-11 · Original record ↗
- arxiv
- 2610.15267
Explicit source versions (1)
- v1 · 2026-10-11
- Preprint available
Source date: 2026-10-11 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Joint-embedding predictive architectures (JEPAs) learn world models directly in latent space, but unconstrained autoregressive predictors may contract the latent distribution over long rollouts. We introduce Symplectic Hamiltonian JEPA (LeHami), which replaces the standard predictor with an action-conditioned Hamiltonian flow over a structured latent state (q, p) integrated using an exactly symplectic leapfrog step. This construction preserves phase-space volume at every rollout step and rules out uniform contraction, asymptotically stable fixed points, and merging of distinct latent states, independently of the learned parameters or action sequence. We obtain the momentum-like component from temporal derivatives of the learned position-like representation, providing an explicit second-order state for prediction and control. On conservative and actuated visual pendulum environments, LeHami substantially improves long-horizon velocity prediction over an unconstrained JEPA world model and empirically maintains constant latent volume where the baseline contracts. The learned flow also supports accurate integration at unseen sub-frame step sizes and numerical time reversal. These results show how geometric structure can provide inference-time guarantees that are complementary to representation regularization in latent world models.