Driving · 2026-10-01
Learning to Plan from Random Exploration
Deqian Kong, Guangyan Sun, Sheng Cheng, Sirui Xie, Bo Pang, Jianwen Xie, Tony Geng, Caiwen Ding, Ying Nian Wu
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
The authors learn multi-horizon temporal relations from random exploration without action or reward labels, using a conditional energy-based model trained with noise-contrastive estimation on observation pairs. A separate one-step local dynamics model predicts candidate action outcomes, and a planner scores progress toward a goal across horizons using learned temporal log-density ratios. They demonstrate long-range planning in mazes from both states and images, and further egocentric navigation and manipulation planning.
Why it matters
The paper investigates whether representations from random (goal-agnostic) trajectories can support long-range planning without policy-improvement training. The approach separates long-range guidance (multi-horizon temporal relations) from local action outcome prediction, enabling closed-loop replanning with both models fixed and providing multiscale “cognitive map” structure consistent with a random-walk analysis.
Method
- Learn horizon-conditioned temporal log-density ratios with a conditional energy-based model using only observation pairs, trained by noise-contrastive estimation (no action/reward labels).
- Train a separate local dynamics model for one-step action outcomes (action-labeled transitions only at horizon τ=1).
- Plan in closed loop: predict candidate action outcomes with the local model, then score progress toward the goal using temporal relations at multiple horizons via greedy (β→0+) or potential-ascent (β→∞) objectives; execute one action then replan with models fixed.
Abstract (from arXiv)
Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.
Related papers
- How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
- EpicWorldModel: Exploration-driven Planning with Latent World Models
- Beyond a single latent space: a dual-latent world model for long-horizon planning
- PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
- Reinforcement Learning for Large-Scale Lifelong Multi-Agent Path Finding with Shared Spatial Fields
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-09-29 · Indexed by Pub-AI: 2026-10-01 UTC
Learning to Plan from Random Exploration
Preprint posted: 2026-09-29 · Original record ↗
- arxiv
- 2609.38383
Explicit source versions (1)
- v1 · 2026-09-29
- Preprint available
Source date: 2026-09-29 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
Random exploration reveals how an environment can be traversed before a goal is specified. Can this experience support long-range planning without policy-improvement training? Our random-walk analysis explains what temporal relations contain: short horizons reveal geodesic geometry in the diffusion limit, while longer horizons reveal connectivity between regions before mixing removes these distinctions. We learn these relations with a conditional energy-based model that estimates temporal log-density ratios through horizon-conditioned embeddings. The model is trained on observation pairs by noise-contrastive estimation, without action or reward labels. The planner queries these learned relations at different horizons as it moves toward the goal. At test time, a separate local dynamics model predicts candidate action outcomes, and the temporal model evaluates their progress toward the goal by selecting or aggregating estimated improvements across horizons. The agent executes one action and replans with both models fixed. Experiments demonstrate long-range maze planning from random exploration using states and images. Learned score fields, embedding probes, and planned routes exhibit properties of a multiscale cognitive map. We further demonstrate egocentric navigation from random exploration and manipulation planning from suboptimal data.
