Latent Dynamics · 2026-11-05

Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models

Ruilong Yu, Fei Ye

arXiv:2611.05847PDFCode

AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.

TL;DR

The authors analyze why a flat JEPA planner’s Euclidean goal cost gives weak temporal guidance over long horizons. They introduce Time-to-Goal (T2G), a lightweight learned temporal metric trained from offline latent trajectories (no rewards/policy labels). With the JEPA encoder/predictor frozen, T2G replaces the Euclidean goal cost and improves long-horizon planning success on multiple benchmarks, approaching or surpassing hierarchical H-JEPA in their evaluations.

Why it matters

Latent planning with JEPAs depends on the goal-matching cost. The paper shows Euclidean latent distances can become less temporally distinguishable at large offsets. By learning an explicit ordering for “time-to-goal” while keeping the world model fixed, the authors isolate and improve the cost geometry used by planners, yielding stronger temporal contrast and higher planning success at long horizons.

Method

  • Analyze flat LeWM’s Euclidean goal cost vs temporal separation, showing conditions where average distance differences shrink at large offsets.
  • Train T2G: a small embedding metric on frozen JEPA latents using latent triples (anchor/near/far) from offline trajectories with a margin ranking loss; no rewards or policy labels.
  • During planning, keep the JEPA encoder/predictor frozen and replace the Euclidean goal cost with T2G distances (last-n or dense aggregation) to score rollout candidates.

Limitation

The dense T2G variant (aggregating temporal distance across the full rollout) does not make planning consistently effective, e.g., it reduces Cube success to 4.0%.

Abstract (from arXiv)

Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to guide temporal progress, states closer in time to the goal should receive lower costs. However, training representations for prediction and distribution regularization does not directly enforce this ordering. We examine this mismatch in flat LeWorldModel (LeWM) by analyzing how its Euclidean goal cost changes with temporal separation. Our analysis identifies conditions under which states at different long offsets receive similar average costs, weakening their temporal distinction. To learn the missing ordering explicitly, we introduce Time-to-Goal (T2G), a lightweight metric trained on latent triples from offline trajectories. A margin ranking loss encourages smaller distances for temporally nearer states, requiring no rewards or policy labels. The learned metric then replaces the Euclidean planning cost while the JEPA encoder and predictor remain frozen. Diagnostics on AntMaze and FourRoom show that T2G preserves stronger temporal contrast and improves near--far ranking at long offsets. Across four navigation and manipulation benchmarks, T2G improves the average planning success rate from 32.7\% (flat LeWM) to 63.0\% and approaches or even surpasses the performance of the hierarchical H-JEPA baselines. These results show that learning temporal ordering for the goal cost can substantially improve latent planning even with a flat world model.

Related papers

Publication record

Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.

First public release: 2026-11-04 · Indexed by Pub-AI: 2026-11-05 UTC

preprint

Time-to-Goal: Learned Temporal Metrics for Long-Horizon Planning with JEPA World Models

Ruilong Yu, Fei Ye

Preprint posted: 2026-11-04 · Original record ↗

arxiv
2611.05847
Explicit source versions (1)
  • v1 · 2026-11-04
  1. Preprint available

    Source date: 2026-11-04 · First detected: 2026-11-05 UTC · Historical baseline

    Source evidence (2)
Abstract · arxiv

Joint-Embedding Predictive Architectures (JEPAs) support planning by comparing predicted latent states with an encoded goal. For this comparison to guide temporal progress, states closer in time to the goal should receive lower costs. However, training representations for prediction and distribution regularization does not directly enforce this ordering. We examine this mismatch in flat LeWorldModel (LeWM) by analyzing how its Euclidean goal cost changes with temporal separation. Our analysis identifies conditions under which states at different long offsets receive similar average costs, weakening their temporal distinction. To learn the missing ordering explicitly, we introduce Time-to-Goal (T2G), a lightweight metric trained on latent triples from offline trajectories. A margin ranking loss encourages smaller distances for temporally nearer states, requiring no rewards or policy labels. The learned metric then replaces the Euclidean planning cost while the JEPA encoder and predictor remain frozen. Diagnostics on AntMaze and FourRoom show that T2G preserves stronger temporal contrast and improves near--far ranking at long offsets. Across four navigation and manipulation benchmarks, T2G improves the average planning success rate from 32.7\% (flat LeWM) to 63.0\% and approaches or even surpasses the performance of the hierarchical H-JEPA baselines. These results show that learning temporal ordering for the goal cost can substantially improve latent planning even with a flat world model.