Driving · 2026-10-20
InJePA: Intention-First Latent Planning with Coupled JEPA World Models for Visual Navigation
Siming Zhou, Bowen Ma, Ruiran Du, Pengcheng Zheng, Sheng Zheng, Dongshen Han, Zhijie Li, Wei Dong, Fan Mo, Chaoning Zhang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
InJepa is an intention-first visual navigation world model. It couples two JEPA models in a shared latent space: a goal-conditioned JEPA proposes latent intentions, and an action-conditioned JEPA predicts successor latent states. Predicted consequences feed back to generate the next intention and infer the next action within each candidate rollout, forming a closed loop. Evaluated in Habitat MP3D (Clean150), InJepa shows effective multistep navigation and works across three frozen visual encoders; code is released.
Why it matters
The authors argue that prior “action-first” candidate search evaluates proposed action sequences but does not guide stepwise construction of navigation plans. InJepa shifts planning from fixed action candidates to latent intention proposals and uses consequence prediction feedback to progressively construct plans. This provides an explicit mechanism for how world models can lead plan construction rather than only score alternatives, while still retaining candidate evaluation and replanning.
Method
- Two coupled JEPA world models in a shared latent space: Proposal JEPA generates goal-conditioned latent intentions; Forward JEPA predicts successor latent states given actions.
- Within each candidate, predicted successor states are fed back to the next intention proposal and action inference step (closed loop of intention, action, and consequence).
- Lookahead selection scores each H-step branch using normalized terminal goal distance and maximum relative intention realization error; select k* and execute one step, then replan.
Abstract (from arXiv)
Visual navigation requires goal directed planning informed by predictions of environment dynamics. Representative navigation world model methods propose complete action candidates before predicting consequences for selection or search updates. For each candidate, predictions primarily evaluate proposed actions rather than guide its stepwise construction. To this end, we propose InJepa, an intention first planning paradigm centered on latent states. A goal conditioned JEPA proposes latent intentions, and an action conditioned JEPA continually predicts future latent states. Both operate in a shared latent space to align their predictive representations. Within each candidate, predicted consequences feed into the next intention proposal and action inference, forming a closed loop of intention, action, and consequence. This extends the world model's role beyond evaluation to leading candidate construction through latent intentions. We evaluate InJepa through closed loop image goal navigation in Habitat MP3D. The results show effective multistep navigation through intention guided action generation and consequence feedback. Experiments across encoders further support applicability across visual representations. These findings support latent intentions as the central planning object and show that world models can directly guide plan construction through coordinated reasoning about intentions and consequences. Code is available at https://github.com/zhuxun132/InJepa.
Related papers
- The Planning Limits of Latent World Models
- H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning
- Beyond Action Prediction: Learning Search-Free Planning in Latent Space
- PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
- EpicWorldModel: Exploration-driven Planning with Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-28 · Indexed by Pub-AI: 2026-10-20 UTC
InJePA: Intention-First Latent Planning with Coupled JEPA World Models for Visual Navigation
Preprint posted: 2026-09-28 · Original record ↗
- arxiv
- 2610.23896
Explicit source versions (1)
- v1 · 2026-09-28
- Preprint available
Source date: 2026-09-28 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Visual navigation requires goal directed planning informed by predictions of environment dynamics. Representative navigation world model methods propose complete action candidates before predicting consequences for selection or search updates. For each candidate, predictions primarily evaluate proposed actions rather than guide its stepwise construction. To this end, we propose InJepa, an intention first planning paradigm centered on latent states. A goal conditioned JEPA proposes latent intentions, and an action conditioned JEPA continually predicts future latent states. Both operate in a shared latent space to align their predictive representations. Within each candidate, predicted consequences feed into the next intention proposal and action inference, forming a closed loop of intention, action, and consequence. This extends the world model's role beyond evaluation to leading candidate construction through latent intentions. We evaluate InJepa through closed loop image goal navigation in Habitat MP3D. The results show effective multistep navigation through intention guided action generation and consequence feedback. Experiments across encoders further support applicability across visual representations. These findings support latent intentions as the central planning object and show that world models can directly guide plan construction through coordinated reasoning about intentions and consequences. Code is available at https://github.com/zhuxun132/InJepa.