Latent Dynamics · 2026-11-03
OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
Dujun Nie, Ruiqi Song, Zixuan Wang, Hanqing Wang, Yifan Han, Baiyong Ding, Wenzhao Zheng, Hangbin Wu, Long Chen
AI-assisted summary based on the paper. Consult the linked source for methods and limitations. Benchmark numbers appear after editor verification.
TL;DR
OpenNWM is a generalist navigation world model that learns universal latent actions from large-scale action-free navigation videos, then aligns them with physical motions via action-conditioned post-training. It introduces NavAnywhere (17.5M frames from 15 sources) and evaluates improved visual prediction and downstream navigation performance on in-domain and out-of-domain benchmarks.
Why it matters
The authors target a key limitation of navigation world models: generalization is constrained by limited action-labeled data. By pretraining with action-free videos using learned universal latent actions, OpenNWM aims to improve both perceptual dynamics learning and navigation performance for open-world navigation.
Method
- Learns Universal Navigation Latent Actions (UNLAs) from action-free videos via visual prediction pretraining, then grounds them with physical-motion supervision when available.
- Trains a latent-conditioned navigation world model using a diffusion-parameterized conditional distribution over future visual states, with latent actions serving as a shared interface for multimodal dynamics.
- Performs receding-horizon planning by optimizing over real waypoint-action sequences using action-conditioned future visual rollouts and an energy combining goal similarity and navigation constraints.
Abstract (from arXiv)
Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
Related papers
- ScaleNav: Scalable Internet Video Pretraining for Language-Conditioned Local Navigation
- Native Action-Prior Learning from Videos for World Action Models
- SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation
- G$^2$WAM: World Action Models for Efficient and Geometry Grounded Image-Goal Navigation
- LiteNWM: Efficient Latent World Models for Onboard Visual Navigation in the Wild
Publication record
Bibliographic evidence describes where and when a record appeared. It does not validate scientific claims.
First announced by arXiv: 2026-11-03 · Indexed by Pub-AI: 2026-11-03 UTC
OpenNWM: A Generalist Navigation World Model with Latent Action Pretraining
First announced by arXiv: 2026-11-03 · Original record ↗
- arxiv
- 2611.01844
Explicit source versions (1)
- v1 · Date not supplied
- Preprint available
Source date: 2026-11-03 · First detected: 2026-11-03 UTC · Historical baseline
Source evidence (1)
- Official evidence ↗ · RSS guid=oai:arXiv.org:2611.01844v1; arxiv:announce_type=new; item pubDate (official metadata)
Abstract · arxiv
Navigation world models predict future observations conditioned on navigation actions to support planning, but their generalization across real-world environments remains constrained by the limited coverage of action-labeled data. Action-free navigation videos provide diverse visual experience, yet lack the recorded actions required for action-conditioned training. We introduce OpenNWM, a generalist navigation world model with latent action pretraining. It learns universal latent actions from large-scale action-free videos through visual prediction pretraining, and further aligns them with physical motions via action-conditioned post-training. To support video pretraining, we curate NavAnywhere, a large-scale action-free navigation video dataset comprising 17.5 million visual observations from 15 heterogeneous sources. It captures navigation by ground robots, humans, and drones, with diverse viewpoints and motion patterns across five broad scene types: residential areas, public and commercial spaces, urban settings, parks, and natural landscapes. Experiments demonstrate improved visual prediction on in-domain and out-of-domain benchmarks, together with improved navigation performance. These results show that latent-action pretraining enables diverse, action-free videos to improve dynamics learning and generalization for open-world navigation.
