Driving · 2026-10-14
SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation
Yifei Yang, Zehua Fan, Zike Yan, Zhongyu Li, Yan Wang
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
SkipWorld is a latent skip planning framework for efficient visual navigation. It trains Latent AutoRoll for closed-loop latent rollout, distills it into SkipDistill for long-stride endpoint prediction, and uses Skip-then-Refine Planning to coarse-filter many candidates with the skip model and rerank selected ones with the autoregressive teacher. On RECON, HuRoN, SCAND, and TartanDrive, it reports lower average ATE/RPE and faster planning than NWM.
Why it matters
The authors target two issues in world-model-based visual navigation: (1) training–deployment mismatch from ground-truth vs closed-loop rollouts, and (2) computational cost from evaluating many candidates with step-by-step rollout. SkipWorld aims to preserve accurate trajectory evaluation while reducing redundant rollout computation via latent skip distillation and coarse-to-fine candidate evaluation.
Method
- Latent AutoRoll: train a world model with closed-loop latent rollout by recursively updating the latent context with predicted clean latents (reducing train–inference mismatch).
- SkipDistill: distill the autoregressive teacher into a skip student that directly predicts long-stride future latent endpoints using endpoint distillation over a skip stride.
- Skip-then-Refine Planning: filter N candidates cheaply with the skip model, then rerank the retained candidates with dense Latent AutoRoll rollouts; average top-K actions.
Abstract (from arXiv)
World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive. We propose SkipWorld, a latent skip planning framework for efficient visual navigation. SkipWorld introduces three components: (i) Latent AutoRoll, which trains a world model to recursively roll out future states in latent space, reducing the mismatch between training and closed-loop inference; (ii) SkipDistill, which distills the latent autoregressive model into a skip world model that directly predicts long-stride future latent states; and (iii) Skip-then-Refine Planning, a dual-model coarse-to-fine planner that first filters candidate trajectories with the skip model and then reranks selected candidates with the stronger autoregressive model. This design preserves accurate trajectory evaluation while reducing redundant rollout computation. Experiments on RECON, HuRoN, SCAND, and TartanDrive show that, compared with NWM, SkipWorld reduces the average ATE by 4.2% and RPE by 2.7%, while decreasing planning time by 62.4% with a 2.7-fold speedup.
Related papers
- LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild
- G$^2$WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation
- Pretrained Features as Teachers, Not States: Compact Latent World Models for Visual Planning
- Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning
- How Long, Not How Close: A Learned Temporal Metric for Planning in Latent World Models
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First public release: 2026-10-13 · Indexed by Pub-AI: 2026-10-14 UTC
SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation
Preprint posted: 2026-10-13 · Original record ↗
- arxiv
- 2610.18135
Explicit source versions (1)
- v1 · 2026-10-13
- Preprint available
Source date: 2026-10-13 · First detected: 2026-10-25 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · atom:published (official metadata)
Abstract · arxiv
World-model-based visual navigation enables action planning by predicting future states under candidate trajectories. However, long-horizon rollout remains challenging: models are often trained with ground-truth visual histories but deployed in closed-loop prediction, and evaluating many candidates with step-by-step rollout is computationally expensive. We propose SkipWorld, a latent skip planning framework for efficient visual navigation. SkipWorld introduces three components: (i) Latent AutoRoll, which trains a world model to recursively roll out future states in latent space, reducing the mismatch between training and closed-loop inference; (ii) SkipDistill, which distills the latent autoregressive model into a skip world model that directly predicts long-stride future latent states; and (iii) Skip-then-Refine Planning, a dual-model coarse-to-fine planner that first filters candidate trajectories with the skip model and then reranks selected candidates with the stronger autoregressive model. This design preserves accurate trajectory evaluation while reducing redundant rollout computation. Experiments on RECON, HuRoN, SCAND, and TartanDrive show that, compared with NWM, SkipWorld reduces the average ATE by 4.2% and RPE by 2.7%, while decreasing planning time by 62.4% with a 2.7-fold speedup.