Latent Dynamics · 2026-10-06
How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
Lama Moukheiber, Haotian Xue, Yongxin Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
TEMPO is a temporal-distance planning objective for frozen latent world-model planners (e.g., LeWM, PLDM). Instead of retraining the world model, it learns a small metric on the frozen latent space where distance reflects the number of environment steps between states, and blends this into the planner’s cost. On 5 tasks (1–3 plans from the goal), it raises LeWM on TwoRoom from 36% to 99% for three plans away.
Why it matters
Latent-distance planning with frozen JEPA world models collapses when the goal is multiple short plans away because latent similarity measures “how close” the end state looks, not “how much time/steps remain” to reach the goal. TEMPO fixes this by adding a temporal progress cost while keeping the world model frozen, enabling short-horizon re-planning to work for long-horizon goals without changing the predictor or planner search mechanics.
Method
- Learn a small MLP F_ϕ on top of the frozen latent z so that distance between two states in an episode reflects the number of environment steps between them (trained by regression on expert frame pairs).
- During planning with CEM, score candidate action sequences with a blended cost: raw latent distance to the goal plus a temporal-distance term ||F_ϕ(ẑ_{t+H})−F_ϕ(G)||² (blended with latent distance; w=0.5).
- Leaves the world model untouched and adds negligible compute: TEMPO requires one small MLP evaluation per candidate and increases per-plan arithmetic by at most 0.3%.
Limitation
TEMPO’s temporal metric is calibrated up to three plans: pairs more than 75 steps apart are not regressed to an exact distance, and the hinge only pushes them at least 3 units apart, so a state more than three plans from the goal never appears closer than a state within three plans.
Abstract (from arXiv)
Latent world models plan by rolling a frozen predictor forward under candidate action sequences and ranking the candidates by the latent distance between their imagined end state and the goal. However, this ranking breaks down when the goal lies several plans away, because the latent distance measures how closely an end state resembles the goal rather than how far it remains from reaching it. To address this, we propose TEMPO, a temporal-distance planning objective that leaves the world model untouched, learns only from the demonstrations already used to train it, and adds negligible cost to the planner's search. TEMPO learns a small map of the frozen latent in which the distance between two states of an episode reflects the number of environment steps between them, and blends this distance into the planner's cost. It requires no rewards, policies or success labels and, being a cost rather than a model, applies to frozen world models with one latent vector per state that plan by a latent distance. We evaluate TEMPO on eleven simulated environments (e.g., maze navigation, tabletop pushing, robotic arm control and three-dimensional manipulation) with the LeWM and PLDM planners. With a small MLP that adds at most 0.3% to a plan's arithmetic, TEMPO improves both planners at every goal distance, including the one-plan setting of their evaluations, raises LeWM from 36% to 99% on TwoRoom three plans from the goal, and remains competitive on a broad range of 2D and 3D navigation, reaching and manipulation tasks.
Related papers
- Beyond a single latent space: a dual-latent world model for long-horizon planning
- The Planning Limits of Latent World Models
- Latent World Models Should Plan with Distances Along the Latent Manifold
- FlexiWorld: Learning and Planning via Flexible Action Chunks Across Multiple Time Scales
- Order-Consistent Latent Geometry for Reliable Short-Horizon Planning in World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-04 · Indexed by Pub-AI: 2026-10-06 UTC
How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
Preprint posted: 2026-10-04 · Original record ↗
- arxiv
- 2610.04988
Explicit source versions (1)
- v2 · 2026-10-07
- Explicit source version detected · v2
Source date: 2026-10-07 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:id explicit version (official metadata)
- Preprint available
Source date: 2026-10-04 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Latent world models plan by rolling a frozen predictor forward under candidate action sequences and ranking the candidates by the latent distance between their imagined end state and the goal. However, this ranking breaks down when the goal lies several plans away, because the latent distance measures how closely an end state resembles the goal rather than how far it remains from reaching it. To address this, we propose TEMPO, a temporal-distance planning objective that leaves the world model untouched, learns only from the recorded trajectories already used to train it, and adds negligible cost to the planner's search. TEMPO learns a small map of the frozen latent in which the distance between two states of an episode reflects the number of environment steps between them, and blends this distance into the planner's cost. It requires no rewards, policies or success labels and, being a cost rather than a model, applies to frozen world models with one latent vector per state that plan by a latent distance. We evaluate TEMPO on eleven simulated environments (e.g., maze navigation, tabletop pushing, robotic arm control and three-dimensional manipulation) with the LeWM and PLDM planners. With a small MLP that adds at most 0.3% to a plan's arithmetic, TEMPO improves both planners at every goal distance, including the one-plan setting of their evaluations, raises LeWM from 36% to 99% on TwoRoom three plans from the goal, and remains competitive on a broad range of 2D and 3D navigation, reaching and manipulation tasks.