Latent Dynamics · 2026-11-05
Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models
Ruilong Yu, Fei Ye
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors analyze why a flat JEPA planner’s Euclidean goal cost gives weak temporal guidance over long horizons. They introduce Time-to-Goal (T2G), a lightweight learned temporal metric trained from offline latent trajectories (no rewards/policy labels). With the JEPA encoder/predictor frozen, T2G replaces the Euclidean goal cost and improves long-horizon planning success on multiple benchmarks, approaching or surpassing hierarchical H-JEPA in their evaluations.
Why it matters
Latent planning with JEPAs depends on the goal-matching cost. The paper shows Euclidean latent distances can become less temporally distinguishable at large offsets. By learning an explicit ordering for “time-to-goal” while keeping the world model fixed, the authors isolate and improve the cost geometry used by planners, yielding stronger temporal contrast and higher planning success at long horizons.
Method
- Analyze flat LeWM’s Euclidean goal cost vs temporal separation, showing conditions where average distance differences shrink at large offsets.
- Train T2G: a small embedding metric on frozen JEPA latents using latent triples (anchor/near/far) from offline trajectories with a margin ranking loss; no rewards or policy labels.
- During planning, keep the JEPA encoder/predictor frozen and replace the Euclidean goal cost with T2G distances (last-n or dense aggregation) to score rollout candidates.
Limitation
The dense T2G variant (aggregating temporal distance across the full rollout) does not make planning consistently effective, e.g., it reduces Cube success to 4.0%.
Abstract (from arXiv)
Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to guide temporal progress, states closer in time to the goal should receive lower costs. However, training representations for prediction and distribution regularization does not directly enforce this ordering. We examine this mismatch in flat LeWorldModel (LeWM) by analyzing how its Euclidean goal cost changes with temporal separation. Our analysis identifies conditions under which states at different long offsets receive similar average costs, weakening their temporal distinction. To learn the missing ordering explicitly, we introduce Time-to-Goal (T2G), a lightweight metric trained on latent triples from offline trajectories. A margin ranking loss encourages smaller distances for temporally nearer states, requiring no rewards or policy labels. The learned metric then replaces the Euclidean planning cost while the JEPA encoder and predictor remain frozen. Diagnostics on AntMaze and FourRoom show that T2G preserves stronger temporal contrast and improves near--far ranking at long offsets. Across four navigation and manipulation benchmarks, T2G improves the average planning success rate from 32.7\% (flat LeWM) to 63.0\% and approaches or even surpasses the performance of the hierarchical H-JEPA baselines. These results show that learning temporal ordering for the goal cost can substantially improve latent planning even with a flat world model.
Related papers
- How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
- Beyond a single latent space: a dual-latent world model for long-horizon planning
- H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
- JEPA-TTT: Persistent Test-Time Training of Latent World Models for Planning under Dynamics Shifts
- ATLAS: Aligned Transport of Latent Structure for Reliable World Model Planning
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-11-04 · Indexed by Pub-AI: 2026-11-05 UTC
Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models
Preprint posted: 2026-11-04 · Original record ↗
- arxiv
- 2611.05847
Explicit source versions (1)
- v1 · 2026-11-04
- Preprint available
Source date: 2026-11-04 · First detected: 2026-11-05 UTC · Historical baseline
Source evidence (2)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.05847v1; arxiv:announce_type=new; item pubDate (official metadata)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to guide temporal progress, states closer in time to the goal should receive lower costs. However, training representations for prediction and distribution regularization does not directly enforce this ordering. We examine this mismatch in flat LeWorldModel (LeWM) by analyzing how its Euclidean goal cost changes with temporal separation. Our analysis identifies conditions under which states at different long offsets receive similar average costs, weakening their temporal distinction. To learn the missing ordering explicitly, we introduce Time-to-Goal (T2G), a lightweight metric trained on latent triples from offline trajectories. A margin ranking loss encourages smaller distances for temporally nearer states, requiring no rewards or policy labels. The learned metric then replaces the Euclidean planning cost while the JEPA encoder and predictor remain frozen. Diagnostics on AntMaze and FourRoom show that T2G preserves stronger temporal contrast and improves near--far ranking at long offsets. Across four navigation and manipulation benchmarks, T2G improves the average planning success rate from 32.7\% (flat LeWM) to 63.0\% and approaches or even surpasses the performance of the hierarchical H-JEPA baselines. These results show that learning temporal ordering for the goal cost can substantially improve latent planning even with a flat world model.
